From a5ecb8df38e1e0cee16ec70403aec8614bb28f18 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 20 Apr 2023 08:29:28 +0200 Subject: [PATCH 001/190] Merging 1.x down into devel (#269) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_sync/database_sync-mysql/tasks/sync.yml | 4 ++-- roles/sync/files_sync/tasks/sync.yml | 10 +++++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index c695982c..9b949f27 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -37,7 +37,7 @@ - name: Register remote dump name (from database). ansible.builtin.set_fact: - mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}.sql.bz2" + mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.bz2" - name: Get source last known good build number. ansible.builtin.command: @@ -91,7 +91,7 @@ - name: Register tmp target dump name. ansible.builtin.set_fact: - mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}.sql.bz2" + mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.bz2" - name: Get target last known good build number. ansible.builtin.command: diff --git a/roles/sync/files_sync/tasks/sync.yml b/roles/sync/files_sync/tasks/sync.yml index e2160008..58ca4cd0 100644 --- a/roles/sync/files_sync/tasks/sync.yml +++ b/roles/sync/files_sync/tasks/sync.yml @@ -1,7 +1,11 @@ --- +- name: Register file sync location. + ansible.builtin.set_fact: + file_sync_path: "{{ files.source.temp_dir }}/{{ files.source.build_id }}_{{ build_number }}" + - name: Create a temporary directory for source files on localhost. ansible.builtin.file: - path: "{{ files.source.temp_dir }}/{{ files.source.build_id }}" + path: "{{ file_sync_path }}" state: directory owner: "{{ deploy_user }}" group: "{{ deploy_user }}" @@ -10,12 +14,12 @@ - name: Copy the source files onto the deploy server. ansible.builtin.command: - cmd: "rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -aHPv {{ files.source.host }}:{{ files.source.files_dir }}/ {{ files.source.temp_dir }}/{{ files.source.build_id }}/" + cmd: "rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -aHPv {{ files.source.host }}:{{ files.source.files_dir }}/ {{ file_sync_path }}/" delegate_to: "localhost" run_once: true - name: Copy the source files from the deploy server onto the destination server. ansible.builtin.command: - cmd: "rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -aHPv {{ files.source.temp_dir }}/{{ files.source.build_id }}/ {{ ansible_play_hosts[0] }}:{{ files.target.files_dir }}/" + cmd: "rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -aHPv {{ file_sync_path }}/ {{ ansible_play_hosts[0] }}:{{ files.target.files_dir }}/" delegate_to: "localhost" run_once: true From acb35fefb1067af7f04a4b8fde1d421d4d8a4e7f Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 21 Apr 2023 17:23:56 +0200 Subject: [PATCH 002/190] Target host checking pr devel (#270) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- scripts/_common.sh | 24 ++++++++++++++++++++++++ scripts/build.sh | 4 ++++ scripts/host-check.yml | 15 +++++++++++++++ 3 files changed, 43 insertions(+) create mode 100644 scripts/host-check.yml diff --git a/scripts/_common.sh b/scripts/_common.sh index 3715c97a..63e89bb9 100755 --- a/scripts/_common.sh +++ b/scripts/_common.sh @@ -9,6 +9,7 @@ export ANSIBLE_CONFIG="$OWN_DIR/ansible.cfg" TARGET_DEPLOY_REPO="" TARGET_DEPLOY_PLAYBOOK="" TARGET_DEPLOY_BRANCH="" +TARGET_DEPLOY_HOST="" PREVIOUS_BUILD_NUMBER="" CURRENT_BUILD_NUMBER="" ANSIBLE_EXTRA_VARS="" @@ -49,6 +50,10 @@ parse_options(){ shift TARGET_DEPLOY_PLAYBOOK="$1" ;; + "--host") + shift + TARGET_DEPLOY_HOST="$1" + ;; "--build-number") shift CURRENT_BUILD_NUMBER="$1" @@ -146,6 +151,25 @@ cleanup_build_tmp_dir(){ fi } +# Call Ansible playbook to ensure host exists. +ansible_host_check(){ + if [ -n "$TARGET_DEPLOY_HOST" ]; then + ANSIBLE_BIN=$(command -v ansible-playbook) + ANSIBLE_CMD="$ANSIBLE_BIN $OWN_DIR/scripts/host-check.yml" + if [ "$DRY_RUN" = "yes" ]; then + ANSIBLE_CMD="$ANSIBLE_CMD --check" + fi + if [ "$VERBOSE" = "yes" ]; then + ANSIBLE_CMD="$ANSIBLE_CMD -vvvv" + fi + if [ -n "$BOTO_PROFILE" ]; then + export AWS_PROFILE="$BOTO_PROFILE" + fi + $ANSIBLE_CMD --extra-vars "{_deploy_host: $TARGET_DEPLOY_HOST}" --extra-vars "$ANSIBLE_DEFAULT_EXTRA_VARS" --extra-vars "$ANSIBLE_EXTRA_VARS" + return $? + fi +} + # Trigger actual Ansible job. # $1 (string) # Operation to perform. diff --git a/scripts/build.sh b/scripts/build.sh index 5f7fa21b..233ca764 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -19,6 +19,7 @@ usage(){ echo '--branch: The branch to deploy.' echo '' echo 'Available options:' + echo '--host: Valid Ansible hostname, if you want to run a host check. Can also be a group name.' echo '--ansible-extra-vars: Variable to pass as --extra-vars arguments to ansible-playbook. Make sure to escape them properly.' echo '--previous-stable-build-number: an incremental build number that ' echo '--dry-run: Do not perform any action but run the playbooks in --check mode.' @@ -94,6 +95,9 @@ fi # Get Ansible defaults. get_ansible_defaults_vars +# Optionally carry out a host check. +ansible_host_check + # From this point on, we want to trigger the "revert" if anything fails. ANSIBLE_BUILD_RESULT=1 # Trigger deploy. diff --git a/scripts/host-check.yml b/scripts/host-check.yml new file mode 100644 index 00000000..980c63cb --- /dev/null +++ b/scripts/host-check.yml @@ -0,0 +1,15 @@ +- hosts: localhost + connection: local + become: false + tasks: + - name: Ensure the hostname check variable is empty. + ansible.builtin.set_fact: + _ce_deploy_ansible_host_check: "" + - name: Check to see if the Ansible host or hostgroup exists. + ansible.builtin.set_fact: + _ce_deploy_ansible_host_check: "{{ item }}" + with_inventory_hostnames: + - "{{ _deploy_host }}" + - ansible.builtin.fail: + msg: "Host does not exist!" + when: _ce_deploy_ansible_host_check | length == 0 From 984b3c95d1d6561f401598567e1fb3b085c7732f Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 25 Apr 2023 09:43:13 +0200 Subject: [PATCH 003/190] Sync enhancements pr devel (#272) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_sync-mysql/defaults/main.yml | 1 + .../database_sync-mysql/tasks/sync.yml | 20 ++++++++++++++++--- roles/sync/files_sync/defaults/main.yml | 3 +++ roles/sync/files_sync/tasks/sync.yml | 17 ++++++++++++++-- 4 files changed, 36 insertions(+), 5 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index 0ab99e98..48c02b8b 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -1,6 +1,7 @@ --- mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here + cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes databases: - source: # Name of the database to take a dump from. diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 9b949f27..7815868f 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -104,12 +104,12 @@ delegate_to: localhost when: database.target.type == 'rolling' -- name: Register target database name. +- name: Register target rolling database name. ansible.builtin.set_fact: mysql_sync_target_database: "{{ database.target.database }}_{{ mysql_sync_target_build_number.stdout }}" when: database.target.type == 'rolling' -- name: Register target database name. +- name: Register target static database name. ansible.builtin.set_fact: mysql_sync_target_database: "{{ database.target.database }}" when: not database.target.type == 'rolling' @@ -139,11 +139,25 @@ args: executable: /bin/bash -- name: Remove tmp dump file. +- name: Delete temporary dump file on target. ansible.builtin.file: path: "{{ mysql_sync_target_dump_path }}" state: absent +- name: Delete temporary dump file on source. + ansible.builtin.file: + path: "{{ mysql_sync_source_dump_path }}" + state: absent + delegate_to: "{{ database.source.host }}" + +- name: Delete temporary dump file on deploy server. + ansible.builtin.file: + path: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" + state: absent + delegate_to: localhost + when: + - mysql_sync.cleanup + - name: Enable all autoscale processes on source ASG. ansible.builtin.command: > aws autoscaling resume-processes --auto-scaling-group-name {{ database.source.asg }} --region {{ mysql_sync_source_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} diff --git a/roles/sync/files_sync/defaults/main.yml b/roles/sync/files_sync/defaults/main.yml index 5b65b1fb..4e222b40 100644 --- a/roles/sync/files_sync/defaults/main.yml +++ b/roles/sync/files_sync/defaults/main.yml @@ -1,5 +1,8 @@ --- files_sync: + unique_workspace: false # set to true to grab a complete full set of files every sync + # Generally speaking you will *not* want to clean up after file syncs, as leaving the files there makes the next rsync far quicker. + cleanup: false # set to true to delete the synced files after a sync directories: - source: # Location of the files to sync from. DO NOT INCLUDE TRAILING SLASH! diff --git a/roles/sync/files_sync/tasks/sync.yml b/roles/sync/files_sync/tasks/sync.yml index 58ca4cd0..516059c2 100644 --- a/roles/sync/files_sync/tasks/sync.yml +++ b/roles/sync/files_sync/tasks/sync.yml @@ -1,7 +1,12 @@ --- - name: Register file sync location. + ansible.builtin.set_fact: + file_sync_path: "{{ files.source.temp_dir }}/{{ files.source.build_id }}" + +- name: Register unique file sync location. ansible.builtin.set_fact: file_sync_path: "{{ files.source.temp_dir }}/{{ files.source.build_id }}_{{ build_number }}" + when: files_sync.unique_workspace - name: Create a temporary directory for source files on localhost. ansible.builtin.file: @@ -15,11 +20,19 @@ - name: Copy the source files onto the deploy server. ansible.builtin.command: cmd: "rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -aHPv {{ files.source.host }}:{{ files.source.files_dir }}/ {{ file_sync_path }}/" - delegate_to: "localhost" + delegate_to: localhost run_once: true - name: Copy the source files from the deploy server onto the destination server. ansible.builtin.command: cmd: "rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -aHPv {{ file_sync_path }}/ {{ ansible_play_hosts[0] }}:{{ files.target.files_dir }}/" - delegate_to: "localhost" + delegate_to: localhost run_once: true + +- name: Delete synced files on deploy server. + ansible.builtin.file: + path: "{{ file_sync_path }}" + state: absent + delegate_to: localhost + when: + - files_sync.cleanup From 3e2977500ee9fb68c8e5b01e836e07718549ab66 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 26 Apr 2023 09:53:09 +0200 Subject: [PATCH 004/190] Bug fixes pr devel (#274) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../config_generate/config_generate-drupal7/tasks/settings.yml | 2 +- .../config_generate/config_generate-drupal8/tasks/settings.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/config_generate/config_generate-drupal7/tasks/settings.yml b/roles/config_generate/config_generate-drupal7/tasks/settings.yml index f8e3b55e..c49d2d63 100644 --- a/roles/config_generate/config_generate-drupal7/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal7/tasks/settings.yml @@ -19,6 +19,6 @@ src: "{{ item }}" dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/settings.php" with_first_found: - - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php" - "settings.php.j2" diff --git a/roles/config_generate/config_generate-drupal8/tasks/settings.yml b/roles/config_generate/config_generate-drupal8/tasks/settings.yml index ebf182e8..aeb4f56d 100644 --- a/roles/config_generate/config_generate-drupal8/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal8/tasks/settings.yml @@ -23,6 +23,6 @@ src: "{{ item }}" dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/settings.php" with_first_found: - - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php" - "settings.php.j2" From 0e5c6a9e3fae493bb77a894e5c33a24f23e677ee Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 26 Apr 2023 09:57:23 +0200 Subject: [PATCH 005/190] Revert "Bug fixes pr devel (#274)" (#276) This reverts commit 3e2977500ee9fb68c8e5b01e836e07718549ab66. --- .../config_generate/config_generate-drupal7/tasks/settings.yml | 2 +- .../config_generate/config_generate-drupal8/tasks/settings.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/config_generate/config_generate-drupal7/tasks/settings.yml b/roles/config_generate/config_generate-drupal7/tasks/settings.yml index c49d2d63..f8e3b55e 100644 --- a/roles/config_generate/config_generate-drupal7/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal7/tasks/settings.yml @@ -19,6 +19,6 @@ src: "{{ item }}" dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/settings.php" with_first_found: - - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" + - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php" - "settings.php.j2" diff --git a/roles/config_generate/config_generate-drupal8/tasks/settings.yml b/roles/config_generate/config_generate-drupal8/tasks/settings.yml index aeb4f56d..ebf182e8 100644 --- a/roles/config_generate/config_generate-drupal8/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal8/tasks/settings.yml @@ -23,6 +23,6 @@ src: "{{ item }}" dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/settings.php" with_first_found: - - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" + - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php" - "settings.php.j2" From 78e6f1bc0af2bbabffcf46f67edae1f29a744c9b Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 11 May 2023 10:49:57 +0200 Subject: [PATCH 006/190] Bug fixes pr devel (#277) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/sync/database_sync/database_sync-mysql.md | 1 + .../config_generate-drupal7/tasks/settings.yml | 2 +- .../sync/database_sync/database_sync-mysql/README.md | 1 + scripts/_common.sh | 6 +++--- scripts/build.sh | 11 +++++++++-- 5 files changed, 15 insertions(+), 6 deletions(-) diff --git a/docs/roles/sync/database_sync/database_sync-mysql.md b/docs/roles/sync/database_sync/database_sync-mysql.md index cf7e5311..cbe1a353 100644 --- a/docs/roles/sync/database_sync/database_sync-mysql.md +++ b/docs/roles/sync/database_sync/database_sync-mysql.md @@ -6,6 +6,7 @@ Sync MySQL databases between environments. --- mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here + cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes databases: - source: # Name of the database to take a dump from. diff --git a/roles/config_generate/config_generate-drupal7/tasks/settings.yml b/roles/config_generate/config_generate-drupal7/tasks/settings.yml index f8e3b55e..55d9dee7 100644 --- a/roles/config_generate/config_generate-drupal7/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal7/tasks/settings.yml @@ -19,6 +19,6 @@ src: "{{ item }}" dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/settings.php" with_first_found: - - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" + - "{{ playbook_dir }}/{{ site.folder }}/{{ build_type }}.settings.php.j2" - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php" - "settings.php.j2" diff --git a/roles/sync/database_sync/database_sync-mysql/README.md b/roles/sync/database_sync/database_sync-mysql/README.md index cf7e5311..cbe1a353 100644 --- a/roles/sync/database_sync/database_sync-mysql/README.md +++ b/roles/sync/database_sync/database_sync-mysql/README.md @@ -6,6 +6,7 @@ Sync MySQL databases between environments. --- mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here + cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes databases: - source: # Name of the database to take a dump from. diff --git a/scripts/_common.sh b/scripts/_common.sh index 63e89bb9..74ba9a60 100755 --- a/scripts/_common.sh +++ b/scripts/_common.sh @@ -156,9 +156,6 @@ ansible_host_check(){ if [ -n "$TARGET_DEPLOY_HOST" ]; then ANSIBLE_BIN=$(command -v ansible-playbook) ANSIBLE_CMD="$ANSIBLE_BIN $OWN_DIR/scripts/host-check.yml" - if [ "$DRY_RUN" = "yes" ]; then - ANSIBLE_CMD="$ANSIBLE_CMD --check" - fi if [ "$VERBOSE" = "yes" ]; then ANSIBLE_CMD="$ANSIBLE_CMD -vvvv" fi @@ -167,6 +164,9 @@ ansible_host_check(){ fi $ANSIBLE_CMD --extra-vars "{_deploy_host: $TARGET_DEPLOY_HOST}" --extra-vars "$ANSIBLE_DEFAULT_EXTRA_VARS" --extra-vars "$ANSIBLE_EXTRA_VARS" return $? + # No host to check provided, just return a clean exit code. + else + return 0 fi } diff --git a/scripts/build.sh b/scripts/build.sh index 233ca764..fa9a1211 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -95,8 +95,15 @@ fi # Get Ansible defaults. get_ansible_defaults_vars -# Optionally carry out a host check. +# Optionally carry out a host check if --host is provided. +ANSIBLE_HOST_CHECK_RESULT=1 ansible_host_check +ANSIBLE_HOST_CHECK_RESULT=$? +# Exit early if host not found. +if [ -n "$ANSIBLE_HOST_CHECK_RESULT" ] && [ "$ANSIBLE_HOST_CHECK_RESULT" != 0 ]; then + echo "ce-deploy failed to find the host. Aborting." + exit 1 +fi # From this point on, we want to trigger the "revert" if anything fails. ANSIBLE_BUILD_RESULT=1 @@ -113,5 +120,5 @@ if [ -n "$ANSIBLE_BUILD_RESULT" ] && [ "$ANSIBLE_BUILD_RESULT" = 0 ]; then exit 0 fi # Failed somehow. Normally unreachable in strict mode. -echo "Something went wrong. Please fill a bug report against ce-deploy." +echo "Something went unexpectedly wrong with ce-deploy. Please file a bug report - https://github.com/codeenigma/ce-deploy/issues/new" exit 1 \ No newline at end of file From a950d6106a98314ec9c827067e7b69ab97344c3c Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 11 May 2023 10:54:29 +0200 Subject: [PATCH 007/190] Bug fixes pr devel (#279) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- scripts/build.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/scripts/build.sh b/scripts/build.sh index fa9a1211..93f36887 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -96,7 +96,6 @@ fi get_ansible_defaults_vars # Optionally carry out a host check if --host is provided. -ANSIBLE_HOST_CHECK_RESULT=1 ansible_host_check ANSIBLE_HOST_CHECK_RESULT=$? # Exit early if host not found. From 30657690d22079aa781b6e2a7d9bb330ca47047a Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 11 May 2023 13:59:04 +0200 Subject: [PATCH 008/190] Bug fixes pr devel (#280) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_code/deploy_code-custom/tasks/main.yml | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 roles/deploy_code/deploy_code-custom/tasks/main.yml diff --git a/roles/deploy_code/deploy_code-custom/tasks/main.yml b/roles/deploy_code/deploy_code-custom/tasks/main.yml new file mode 100644 index 00000000..6f34c92f --- /dev/null +++ b/roles/deploy_code/deploy_code-custom/tasks/main.yml @@ -0,0 +1,3 @@ +--- + +# Nothing to do here. \ No newline at end of file From 17455776c79c4b3637df918c3d296352315a814f Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 12 May 2023 14:48:56 +0200 Subject: [PATCH 009/190] Bug fixes pr devel (#281) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/_init.md | 1 + roles/_init/README.md | 1 + roles/_init/defaults/main.yml | 1 + roles/_init/tasks/main.yml | 1 + 4 files changed, 4 insertions(+) diff --git a/docs/roles/_init.md b/docs/roles/_init.md index 9cb9e024..08ffe301 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -39,6 +39,7 @@ mautic: bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 +install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app ``` diff --git a/roles/_init/README.md b/roles/_init/README.md index 9cb9e024..08ffe301 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -39,6 +39,7 @@ mautic: bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 +install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app ``` diff --git a/roles/_init/defaults/main.yml b/roles/_init/defaults/main.yml index 9a955f3e..ac06e2a7 100644 --- a/roles/_init/defaults/main.yml +++ b/roles/_init/defaults/main.yml @@ -30,3 +30,4 @@ mautic: bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 +install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index a0d90571..c054e8fb 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -124,3 +124,4 @@ - name: Ensure we have a cachetool binary. ansible.builtin.import_role: name: cli/cachetool + when: install_php_cachetool From 1d3d7408586cdb830a6e336aa0fbc63dac961f10 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 25 May 2023 15:33:40 +0200 Subject: [PATCH 010/190] Bug fixes pr devel (#283) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/_meta/deploy-drupal8/tasks/main.yml | 4 ++-- roles/database_apply/database_apply-drupal7/tasks/main.yml | 5 ----- roles/database_apply/database_apply-drupal8/tasks/main.yml | 5 ----- roles/deploy_code/tasks/cleanup.yml | 6 +++--- 4 files changed, 5 insertions(+), 15 deletions(-) diff --git a/roles/_meta/deploy-drupal8/tasks/main.yml b/roles/_meta/deploy-drupal8/tasks/main.yml index 6ce31d4f..3e66875b 100644 --- a/roles/_meta/deploy-drupal8/tasks/main.yml +++ b/roles/_meta/deploy-drupal8/tasks/main.yml @@ -20,14 +20,14 @@ name: database_backup - import_role: name: config_generate +- import_role: + name: cache_clear/cache_clear-opcache - import_role: name: database_apply - import_role: name: sanitize/admin_creds - import_role: name: live_symlink -- import_role: - name: cache_clear/cache_clear-opcache - import_role: name: cache_clear/cache_clear-drupal8 - import_role: diff --git a/roles/database_apply/database_apply-drupal7/tasks/main.yml b/roles/database_apply/database_apply-drupal7/tasks/main.yml index 4f476253..3012032f 100644 --- a/roles/database_apply/database_apply-drupal7/tasks/main.yml +++ b/roles/database_apply/database_apply-drupal7/tasks/main.yml @@ -26,11 +26,6 @@ name: "cache_clear/cache_clear-{{ project_type }}" when: previous_build_number > 0 -- name: Clear the opcache. - ansible.builtin.include_role: - name: cache_clear/cache_clear-opcache - when: previous_build_number > 0 - - name: Apply Drupal database updates. ansible.builtin.shell: cmd: "{{ drush_bin }} -l {{ site.folder }} -y updb" diff --git a/roles/database_apply/database_apply-drupal8/tasks/main.yml b/roles/database_apply/database_apply-drupal8/tasks/main.yml index 4dc66a3a..c649826c 100644 --- a/roles/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/database_apply/database_apply-drupal8/tasks/main.yml @@ -44,11 +44,6 @@ - previous_build_number > 0 - site.config_import_command != 'deploy' -- name: Clear the opcache. - ansible.builtin.include_role: - name: cache_clear/cache_clear-opcache - when: previous_build_number > 0 - - name: Apply Drupal database updates. ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site.folder }} -y updb" diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index db2bb9bb..b3a4f132 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -150,7 +150,7 @@ - deploy_code.mount_sync is defined - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_check is succeeded + - _deploy_code_mount_check.rc == 0 - deploy_code.services | length > 0 - name: Stop any services that might be keeping the loop device busy. @@ -166,7 +166,7 @@ - deploy_code.mount_sync is defined - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_check is succeeded + - _deploy_code_mount_check.rc == 0 - deploy_code.services | length > 0 - name: Unmount existing SquashFS image. @@ -177,7 +177,7 @@ - deploy_code.mount_sync is defined - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_check is succeeded + - _deploy_code_mount_check.rc == 0 - name: Mount new SquashFS image. ansible.builtin.command: From 585e552c636d6812dc5a8c3b5467ff0841e991c5 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 21 Jun 2023 11:25:01 +0200 Subject: [PATCH 011/190] Verbose drush output pr devel (#288) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/_init.md | 1 + roles/_init/README.md | 1 + roles/_init/defaults/main.yml | 1 + .../cache_clear-drupal8/tasks/main.yml | 9 ++++++++- .../database_apply-drupal8/tasks/main.yml | 19 +++++++++++++++++++ .../database_apply-drupal8/tasks/main.yml | 12 ++++++++++++ 6 files changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/roles/_init.md b/docs/roles/_init.md index 08ffe301..edf1f5d3 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -15,6 +15,7 @@ deploy_user: "deploy" # for MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" drupal: + drush_verbose_output: false sites: - folder: "default" public_files: "sites/default/files" diff --git a/roles/_init/README.md b/roles/_init/README.md index 08ffe301..edf1f5d3 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -15,6 +15,7 @@ deploy_user: "deploy" # for MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" drupal: + drush_verbose_output: false sites: - folder: "default" public_files: "sites/default/files" diff --git a/roles/_init/defaults/main.yml b/roles/_init/defaults/main.yml index ac06e2a7..57ee7e44 100644 --- a/roles/_init/defaults/main.yml +++ b/roles/_init/defaults/main.yml @@ -6,6 +6,7 @@ deploy_user: "deploy" # for MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" drupal: + drush_verbose_output: false sites: - folder: "default" public_files: "sites/default/files" diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index 5b780086..e9524b81 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,6 +1,6 @@ --- - name: Clear Drupal cache. - command: + ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site.folder }} -y cr" chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" @@ -9,3 +9,10 @@ loop_control: loop_var: site run_once: true + register: _drush_output + +- name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: drupal.drush_verbose_output + diff --git a/roles/database_apply/database_apply-drupal8/tasks/main.yml b/roles/database_apply/database_apply-drupal8/tasks/main.yml index c649826c..e2fecd17 100644 --- a/roles/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/database_apply/database_apply-drupal8/tasks/main.yml @@ -23,6 +23,12 @@ loop_control: loop_var: site when: (previous_build_number == 0) or (site.force_install is defined and site.force_install) + register: _drush_output + +- name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: drupal.drush_verbose_output - name: Fix permissions on Drupal directory. ansible.builtin.file: @@ -54,6 +60,12 @@ loop_control: loop_var: site when: site.config_import_command != 'deploy' + register: _drush_output + +- name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: drupal.drush_verbose_output - name: Import configuration. ansible.builtin.command: @@ -67,3 +79,10 @@ when: - previous_build_number > 0 - site.config_import_command + register: _drush_output + +- name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: drupal.drush_verbose_output + diff --git a/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml b/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml index 96d9b65f..d13b0129 100644 --- a/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml @@ -17,6 +17,12 @@ with_items: "{{ drupal.sites }}" loop_control: loop_var: site + register: _drush_output + +- name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: drupal.drush_verbose_output # This only runs if the sync_config_import variable is not defined or it is defined and is true. If it's defined and false, this won't run. - name: Import configuration. @@ -31,6 +37,12 @@ when: - site.config_import_command - site.sync_config_import is not defined or site.sync_config_import + register: _drush_output + +- name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: drupal.drush_verbose_output - name: Clear the cache. ansible.builtin.include_role: From 2efdf2b2c24aa9d5ced6fae9ba4aa88306658b7b Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 21 Jun 2023 14:27:14 +0300 Subject: [PATCH 012/190] Db import speed up pr devel (#290) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../tasks/revert-dump.yml | 16 ++++++++++++++- .../database_sync-mysql/tasks/sync.yml | 20 ++++++++++++++++--- 2 files changed, 32 insertions(+), 4 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index 80898e7d..07cc1530 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -1,7 +1,21 @@ --- +- name: Unpack dump file. + ansible.builtin.unarchive: + src: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.bz2" + dest: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql" + run_once: true + when: previous_build_number > 0 + - name: Revert database from dump. - ansible.builtin.shell: "set -o pipefail && bzcat {{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.bz2 | mysql --defaults-extra-file={{ database.credentials_file }} {{ database.database }}" + ansible.builtin.shell: "mysql --defaults-extra-file={{ database.credentials_file }} {{ database.database }} < /tmp/{{ database.database }}-{{ previous_build_number }}.sql" args: executable: /bin/bash + run_once: true when: previous_build_number > 0 + +- name: Delete unpacked dump file. + ansible.builtin.file: + path: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql" + state: absent run_once: true + when: previous_build_number > 0 diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 7815868f..7bfaea0b 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -93,6 +93,10 @@ ansible.builtin.set_fact: mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.bz2" +- name: Register tmp unpacked target dump name. + ansible.builtin.set_fact: + mysql_sync_target_dump_unpacked_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql" + - name: Get target last known good build number. ansible.builtin.command: argv: @@ -126,6 +130,16 @@ src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" dest: "{{ mysql_sync_target_dump_path }}" +- name: Unpack dump file. + ansible.builtin.unarchive: + src: "{{ mysql_sync_target_dump_path }}" + dest: "{{ mysql_sync_target_dump_unpacked_path }}" + +- name: Delete temporary dump file on target. + ansible.builtin.file: + path: "{{ mysql_sync_target_dump_path }}" + state: absent + - name: Drop target database. ansible.builtin.command: cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'drop database if exists {{ mysql_sync_target_database }};'" @@ -135,13 +149,13 @@ cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'create database {{ mysql_sync_target_database }};'" - name: Repopulate database from dump. - ansible.builtin.shell: "set -o pipefail && bzcat {{ mysql_sync_target_dump_path }} | mysql --defaults-extra-file={{ database.target.credentials_file }} {{ mysql_sync_target_database }}" + ansible.builtin.shell: "mysql --defaults-extra-file={{ database.target.credentials_file }} {{ mysql_sync_target_database }} < {{ mysql_sync_target_dump_unpacked_path }}" args: executable: /bin/bash -- name: Delete temporary dump file on target. +- name: Delete temporary unpacked dump file on target. ansible.builtin.file: - path: "{{ mysql_sync_target_dump_path }}" + path: "{{ mysql_sync_target_dump_unpacked_path }}" state: absent - name: Delete temporary dump file on source. From 8280af4267b0229a9b681608fac6f7a22cc7011e Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 21 Jun 2023 16:56:52 +0300 Subject: [PATCH 013/190] Db import speed up pr devel (#292) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../database_backup/database_backup-mysql/tasks/revert-dump.yml | 1 + roles/sync/database_sync/database_sync-mysql/tasks/sync.yml | 1 + 2 files changed, 2 insertions(+) diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index 07cc1530..38522af3 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -3,6 +3,7 @@ ansible.builtin.unarchive: src: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.bz2" dest: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql" + remote_src: true run_once: true when: previous_build_number > 0 diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 7bfaea0b..11b50d12 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -134,6 +134,7 @@ ansible.builtin.unarchive: src: "{{ mysql_sync_target_dump_path }}" dest: "{{ mysql_sync_target_dump_unpacked_path }}" + remote_src: true - name: Delete temporary dump file on target. ansible.builtin.file: From e9d58a439ab11f2fb31ad2d5dcd31c4e955e3e63 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 21 Jun 2023 18:57:57 +0300 Subject: [PATCH 014/190] Db import speed up pr devel (#293) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../database_sync-mysql/defaults/main.yml | 2 + .../database_sync-mysql/tasks/sync.yml | 63 ++++++++++++++++--- 2 files changed, 55 insertions(+), 10 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index 48c02b8b..72add57e 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -24,6 +24,8 @@ mysql_sync: dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/regular" # If the source is on an ASG, provide the ASG name here. Otherwise, leave empty. asg: "" + # Default is bzip2, set to true if you want to use gzip. + use_gzip: false target: database: "{{ project_name }}_dev" credentials_file: "/home/{{ deploy_user }}/.mysql.creds" diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 11b50d12..e4b040a0 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -35,10 +35,16 @@ - database.target.asg is defined - database.target.asg | length > 0 -- name: Register remote dump name (from database). +- name: Register remote dump name (from database) - BZIP2. ansible.builtin.set_fact: mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.bz2" +- name: Register remote dump name (from database) - GZIP. + ansible.builtin.set_fact: + mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.gz" + when: + - database.source.use_gzip + - name: Get source last known good build number. ansible.builtin.command: argv: @@ -60,7 +66,7 @@ mysql_sync_source_database: "{{ database.source.database }}" when: not database.source.type == 'rolling' -- name: Take a dump from source database. +- name: Take a dump from source database - BZIP2. ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | bzip2 > {{ mysql_sync_source_dump_path }}" args: executable: /bin/bash @@ -68,6 +74,15 @@ when: - database.source.fresh_db +- name: Take a dump from source database - GZIP. + ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | gzip > {{ mysql_sync_source_dump_path }}" + args: + executable: /bin/bash + delegate_to: "{{ database.source.host }}" + when: + - database.source.fresh_db + - database.source.use_gzip + - name: Find source database host. ansible.builtin.command: cmd: "grep 'host' {{ database.source.credentials_file }}" @@ -89,10 +104,16 @@ delegate_to: "{{ database.source.host }}" when: not database.source.fresh_db -- name: Register tmp target dump name. +- name: Register tmp target dump name - BZIP2. ansible.builtin.set_fact: mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.bz2" +- name: Register tmp target dump name - GZIP. + ansible.builtin.set_fact: + mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.gz" + when: + - database.source.use_gzip + - name: Register tmp unpacked target dump name. ansible.builtin.set_fact: mysql_sync_target_dump_unpacked_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql" @@ -118,23 +139,45 @@ mysql_sync_target_database: "{{ database.target.database }}" when: not database.target.type == 'rolling' -- name: Fetch dump file. +- name: Fetch dump file - BZIP2. ansible.builtin.fetch: src: "{{ mysql_sync_source_dump_path }}" dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" flat: true delegate_to: "{{ database.source.host }}" -- name: Copy dump file to destination. +- name: Fetch dump file - GZIP. + ansible.builtin.fetch: + src: "{{ mysql_sync_source_dump_path }}" + dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" + flat: true + delegate_to: "{{ database.source.host }}" + when: + - database.source.use_gzip + +- name: Copy dump file to destination - BZIP2. ansible.builtin.copy: src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" dest: "{{ mysql_sync_target_dump_path }}" -- name: Unpack dump file. - ansible.builtin.unarchive: - src: "{{ mysql_sync_target_dump_path }}" - dest: "{{ mysql_sync_target_dump_unpacked_path }}" - remote_src: true +- name: Copy dump file to destination - GZIP. + ansible.builtin.copy: + src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" + dest: "{{ mysql_sync_target_dump_path }}" + when: + - database.source.use_gzip + +- name: Unpack dump file - BZIP2. + ansible.builtin.shell: "bzip2 -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" + args: + executable: /bin/bash + +- name: Unpack dump file - GZIP. + ansible.builtin.shell: "gzip -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" + args: + executable: /bin/bash + when: + - database.source.use_gzip - name: Delete temporary dump file on target. ansible.builtin.file: From 341234866fbd505e11b1e0fada8fc47fedd104d8 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 21 Jun 2023 19:28:48 +0300 Subject: [PATCH 015/190] Db import speed up pr devel (#294) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../database_sync-mysql/tasks/sync.yml | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index e4b040a0..5be6f8a9 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -42,8 +42,7 @@ - name: Register remote dump name (from database) - GZIP. ansible.builtin.set_fact: mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.gz" - when: - - database.source.use_gzip + when: database.source.use_gzip - name: Get source last known good build number. ansible.builtin.command: @@ -111,8 +110,7 @@ - name: Register tmp target dump name - GZIP. ansible.builtin.set_fact: mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.gz" - when: - - database.source.use_gzip + when: database.source.use_gzip - name: Register tmp unpacked target dump name. ansible.builtin.set_fact: @@ -152,8 +150,7 @@ dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" flat: true delegate_to: "{{ database.source.host }}" - when: - - database.source.use_gzip + when: database.source.use_gzip - name: Copy dump file to destination - BZIP2. ansible.builtin.copy: @@ -164,8 +161,7 @@ ansible.builtin.copy: src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" dest: "{{ mysql_sync_target_dump_path }}" - when: - - database.source.use_gzip + when: database.source.use_gzip - name: Unpack dump file - BZIP2. ansible.builtin.shell: "bzip2 -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" @@ -176,8 +172,7 @@ ansible.builtin.shell: "gzip -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" args: executable: /bin/bash - when: - - database.source.use_gzip + when: database.source.use_gzip - name: Delete temporary dump file on target. ansible.builtin.file: From f46a80948f2b0b6d2c7ca72075edd9584ccca883 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 21 Jun 2023 19:40:28 +0300 Subject: [PATCH 016/190] Db import speed up pr devel (#295) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../database_sync-mysql/defaults/main.yml | 3 +-- .../database_sync-mysql/tasks/sync.yml | 17 +++++++++++------ 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index 72add57e..763a12ec 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -2,6 +2,7 @@ mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes + use_gzip: false # Default is bzip2, set to true if you want to use databases: - source: # Name of the database to take a dump from. @@ -24,8 +25,6 @@ mysql_sync: dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/regular" # If the source is on an ASG, provide the ASG name here. Otherwise, leave empty. asg: "" - # Default is bzip2, set to true if you want to use gzip. - use_gzip: false target: database: "{{ project_name }}_dev" credentials_file: "/home/{{ deploy_user }}/.mysql.creds" diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 5be6f8a9..8075537e 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -42,7 +42,8 @@ - name: Register remote dump name (from database) - GZIP. ansible.builtin.set_fact: mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.gz" - when: database.source.use_gzip + when: + - mysql_sync.use_gzip - name: Get source last known good build number. ansible.builtin.command: @@ -80,7 +81,7 @@ delegate_to: "{{ database.source.host }}" when: - database.source.fresh_db - - database.source.use_gzip + - mysql_sync.use_gzip - name: Find source database host. ansible.builtin.command: @@ -110,7 +111,8 @@ - name: Register tmp target dump name - GZIP. ansible.builtin.set_fact: mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.gz" - when: database.source.use_gzip + when: + - mysql_sync.use_gzip - name: Register tmp unpacked target dump name. ansible.builtin.set_fact: @@ -150,7 +152,8 @@ dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" flat: true delegate_to: "{{ database.source.host }}" - when: database.source.use_gzip + when: + - mysql_sync.use_gzip - name: Copy dump file to destination - BZIP2. ansible.builtin.copy: @@ -161,7 +164,8 @@ ansible.builtin.copy: src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" dest: "{{ mysql_sync_target_dump_path }}" - when: database.source.use_gzip + when: + - mysql_sync.use_gzip - name: Unpack dump file - BZIP2. ansible.builtin.shell: "bzip2 -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" @@ -172,7 +176,8 @@ ansible.builtin.shell: "gzip -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" args: executable: /bin/bash - when: database.source.use_gzip + when: + - mysql_sync.use_gzip - name: Delete temporary dump file on target. ansible.builtin.file: From 5701a3414d4099da34aee5a1beee44e4c3051d48 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 21 Jun 2023 19:49:55 +0300 Subject: [PATCH 017/190] Db import speed up pr devel (#296) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../database_sync/database_sync-mysql/defaults/main.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index 763a12ec..d4866c2c 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -1,8 +1,8 @@ --- mysql_sync: - mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here - cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes - use_gzip: false # Default is bzip2, set to true if you want to use + mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here. + cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes. + use_gzip: false # if true gzip will be used instead of default bzip2. databases: - source: # Name of the database to take a dump from. From 723fb110dce93238397081975675f42b023f4f99 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 21 Jun 2023 20:15:34 +0300 Subject: [PATCH 018/190] Db import speed up pr devel (#297) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../database_sync-mysql/tasks/sync.yml | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 8075537e..9cf2b250 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -38,6 +38,8 @@ - name: Register remote dump name (from database) - BZIP2. ansible.builtin.set_fact: mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.bz2" + when: + - not mysql_sync.use_gzip - name: Register remote dump name (from database) - GZIP. ansible.builtin.set_fact: @@ -73,6 +75,7 @@ delegate_to: "{{ database.source.host }}" when: - database.source.fresh_db + - not mysql_sync.use_gzip - name: Take a dump from source database - GZIP. ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | gzip > {{ mysql_sync_source_dump_path }}" @@ -107,6 +110,8 @@ - name: Register tmp target dump name - BZIP2. ansible.builtin.set_fact: mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.bz2" + when: + - not mysql_sync.use_gzip - name: Register tmp target dump name - GZIP. ansible.builtin.set_fact: @@ -145,6 +150,8 @@ dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" flat: true delegate_to: "{{ database.source.host }}" + when: + - not mysql_sync.use_gzip - name: Fetch dump file - GZIP. ansible.builtin.fetch: @@ -159,6 +166,8 @@ ansible.builtin.copy: src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" dest: "{{ mysql_sync_target_dump_path }}" + when: + - not mysql_sync.use_gzip - name: Copy dump file to destination - GZIP. ansible.builtin.copy: @@ -171,6 +180,8 @@ ansible.builtin.shell: "bzip2 -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" args: executable: /bin/bash + when: + - not mysql_sync.use_gzip - name: Unpack dump file - GZIP. ansible.builtin.shell: "gzip -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" @@ -210,11 +221,14 @@ - name: Delete temporary dump file on deploy server. ansible.builtin.file: - path: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" + path: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql{{ item }}" state: absent delegate_to: localhost when: - mysql_sync.cleanup + with_items: + - ".bz2" + - ".gz" - name: Enable all autoscale processes on source ASG. ansible.builtin.command: > From 59aad1c4de5ccff38f9d3b2c4232a718fd0a3245 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 22 Jun 2023 11:07:04 +0300 Subject: [PATCH 019/190] Db import speed up pr devel (#298) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../database_sync-mysql/defaults/main.yml | 2 +- .../database_sync-mysql/tasks/sync.yml | 86 +++++-------------- 2 files changed, 24 insertions(+), 64 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index d4866c2c..88143a7b 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -2,7 +2,7 @@ mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here. cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes. - use_gzip: false # if true gzip will be used instead of default bzip2. + archival_method: "bzip2" # oprions are "bzip2" or "gzip". databases: - source: # Name of the database to take a dump from. diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 9cf2b250..ac346444 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -35,17 +35,24 @@ - database.target.asg is defined - database.target.asg | length > 0 -- name: Register remote dump name (from database) - BZIP2. + +- name: Register bzip2 archive type vars. ansible.builtin.set_fact: - mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.bz2" + archive_file_type: "bz2" + archival_command: "bzip2" when: - - not mysql_sync.use_gzip + - mysql_sync.archival_method == 'bzip2' -- name: Register remote dump name (from database) - GZIP. +- name: Register gunzip archive type vars. ansible.builtin.set_fact: - mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.gz" + archive_file_type: "gz" + archival_command: "gzip" when: - - mysql_sync.use_gzip + - mysql_sync.archival_method == 'gzip' + +- name: Register remote dump name (from database). + ansible.builtin.set_fact: + mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.{{ archive_file_type }}" - name: Get source last known good build number. ansible.builtin.command: @@ -68,23 +75,13 @@ mysql_sync_source_database: "{{ database.source.database }}" when: not database.source.type == 'rolling' -- name: Take a dump from source database - BZIP2. - ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | bzip2 > {{ mysql_sync_source_dump_path }}" +- name: Take a dump from source database. + ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | {{ archival_command }} > {{ mysql_sync_source_dump_path }}" args: executable: /bin/bash delegate_to: "{{ database.source.host }}" when: - database.source.fresh_db - - not mysql_sync.use_gzip - -- name: Take a dump from source database - GZIP. - ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | gzip > {{ mysql_sync_source_dump_path }}" - args: - executable: /bin/bash - delegate_to: "{{ database.source.host }}" - when: - - database.source.fresh_db - - mysql_sync.use_gzip - name: Find source database host. ansible.builtin.command: @@ -107,17 +104,9 @@ delegate_to: "{{ database.source.host }}" when: not database.source.fresh_db -- name: Register tmp target dump name - BZIP2. +- name: Register tmp target dump name. ansible.builtin.set_fact: - mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.bz2" - when: - - not mysql_sync.use_gzip - -- name: Register tmp target dump name - GZIP. - ansible.builtin.set_fact: - mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.gz" - when: - - mysql_sync.use_gzip + mysql_sync_target_dump_path: "/tmp/{{ database.target.database }}_{{ build_number }}_target.sql.{{ archive_file_type }}" - name: Register tmp unpacked target dump name. ansible.builtin.set_fact: @@ -144,51 +133,22 @@ mysql_sync_target_database: "{{ database.target.database }}" when: not database.target.type == 'rolling' -- name: Fetch dump file - BZIP2. - ansible.builtin.fetch: - src: "{{ mysql_sync_source_dump_path }}" - dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" - flat: true - delegate_to: "{{ database.source.host }}" - when: - - not mysql_sync.use_gzip - -- name: Fetch dump file - GZIP. +- name: Fetch dump file. ansible.builtin.fetch: src: "{{ mysql_sync_source_dump_path }}" - dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" + dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.{{ archive_file_type }}" flat: true delegate_to: "{{ database.source.host }}" - when: - - mysql_sync.use_gzip -- name: Copy dump file to destination - BZIP2. +- name: Copy dump file to destination. ansible.builtin.copy: - src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.bz2" + src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.{{ archive_file_type }}" dest: "{{ mysql_sync_target_dump_path }}" - when: - - not mysql_sync.use_gzip -- name: Copy dump file to destination - GZIP. - ansible.builtin.copy: - src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.gz" - dest: "{{ mysql_sync_target_dump_path }}" - when: - - mysql_sync.use_gzip - -- name: Unpack dump file - BZIP2. - ansible.builtin.shell: "bzip2 -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" +- name: Unpack dump file. + ansible.builtin.shell: "{{ archival_command }} -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" args: executable: /bin/bash - when: - - not mysql_sync.use_gzip - -- name: Unpack dump file - GZIP. - ansible.builtin.shell: "gzip -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" - args: - executable: /bin/bash - when: - - mysql_sync.use_gzip - name: Delete temporary dump file on target. ansible.builtin.file: From 4e17c10bb7217d937b036fbc80cf5ffa8ee1aa34 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 22 Jun 2023 17:35:49 +0200 Subject: [PATCH 020/190] Ecs deployments pr devel (#299) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/_Sidebar.md | 1 + docs/roles/deploy_container.md | 28 +++++++++++ roles/deploy_container/README.md | 30 ++++++++++++ roles/deploy_container/defaults/main.yml | 15 ++++++ roles/deploy_container/tasks/main.yml | 49 +++++++++++++++++++ .../deploy_container/templates/Dockerfile.j2 | 7 +++ 6 files changed, 130 insertions(+) create mode 100644 docs/roles/deploy_container.md create mode 100644 roles/deploy_container/README.md create mode 100644 roles/deploy_container/defaults/main.yml create mode 100644 roles/deploy_container/tasks/main.yml create mode 100644 roles/deploy_container/templates/Dockerfile.j2 diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index 16139d8f..3d47469a 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -31,6 +31,7 @@ - [Data backups](/roles/database_backup) - [MySQL backups](/roles/database_backup/database_backup-mysql) - [Deploy](/roles/deploy_code) + - [Deploy container](/roles/deploy_container) - [Init](/roles/_init) - [LHCI run](/roles/lhci_run) - ["Meta"](/roles/_meta) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md new file mode 100644 index 00000000..46285580 --- /dev/null +++ b/docs/roles/deploy_container.md @@ -0,0 +1,28 @@ +# Deploy container +Step that deploys the codebase in a Docker container image. + + + + + +## Default variables +```yaml +--- +deploy_container: + container_name: "example/example" + container_tag: latest # tag will take format container_name:container_tag + docker_registry_url: https://index.docker.io/v1/ + docker_registry_user: example + docker_registry_pass: asdf1234 + docker_base_command: "docker image build" + docker_build_dir: "{{ _ce_deploy_build_dir }}" + environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template + # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached + aws_ecr: + enabled: false # set to true if using AWS ECR + region: eu-west-1 + profile: example + +``` + + diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md new file mode 100644 index 00000000..0647af50 --- /dev/null +++ b/roles/deploy_container/README.md @@ -0,0 +1,30 @@ +# Deploy container +Step that deploys the codebase in a Docker container image. Requires Docker and the `community.docker` collection for Ansible to be installed on your deploy server. This can be handled by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. + +AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the managed AWS `EC2InstanceProfileForImageBuilderECRContainerBuilds` policy attached via IAM to allow access to fetch credentials and push containers. + + + + + +## Default variables +```yaml +--- +deploy_container: + container_name: "example/example" + container_tag: latest # tag will take format container_name:container_tag + docker_registry_url: https://index.docker.io/v1/ + docker_registry_user: example + docker_registry_pass: asdf1234 + docker_base_command: "docker image build" + docker_build_dir: "{{ _ce_deploy_build_dir }}" + environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template + # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached + aws_ecr: + enabled: false # set to true if using AWS ECR + region: eu-west-1 + profile: example + +``` + + diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml new file mode 100644 index 00000000..4e4d30c3 --- /dev/null +++ b/roles/deploy_container/defaults/main.yml @@ -0,0 +1,15 @@ +--- +deploy_container: + container_name: "example/example" + container_tag: latest # tag will take format container_name:container_tag + docker_registry_url: https://index.docker.io/v1/ + docker_registry_user: example + docker_registry_pass: asdf1234 + docker_base_command: "docker image build" + docker_build_dir: "{{ _ce_deploy_build_dir }}" + environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template + # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached + aws_ecr: + enabled: false # set to true if using AWS ECR + region: eu-west-1 + profile: example diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml new file mode 100644 index 00000000..4eb9d87a --- /dev/null +++ b/roles/deploy_container/tasks/main.yml @@ -0,0 +1,49 @@ +--- +# @TODO - for AWS ECR we'll need certain policies attaching to the deploy IAM user +- name: Create Dockerfile from template. + local_action: + module: ansible.builtin.template + src: Dockerfile.j2 + dest: "{{ deploy_container.docker_build_dir }}/Dockerfile" + +- name: Set Docker registry username and password. + ansible.builtin.set_fact: + _docker_registry_username: "{{ deploy_container.docker_registry_user }}" + _docker_registry_password: "{{ deploy_container.docker_registry_pass }}" + +# Token valid for 12 hours +- name: Fetch AWS ECR registry login token. + ansible.builtin.command: + command: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.profile }}" + when: deploy_container.aws_ecr.enabled + register: _docker_registry_ecr_token + +- name: Set AWS ECR registry password. + ansible.builtin.set_fact: + _docker_registry_password: "{{ _docker_registry_ecr_token.stdout }}" + when: deploy_container.aws_ecr.enabled + +- name: Set AWS ECR registry username. + ansible.builtin.set_fact: + _docker_registry_username: "AWS" + when: deploy_container.aws_ecr.enabled + +- name: Log into Docker registry. + community.docker.docker_login: + registry_url: "{{ deploy_container.docker_registry_url }}" + username: "{{ _docker_registry_username }}" + password: "{{ _docker_registry_password }}" + reauthorize: true + delegate_to: localhost + when: deploy_container.docker_registry_login + +- name: Build and push container image. + community.docker.docker_image: + build: + path: "{{ deploy_container.docker_build_dir }}" + repository: "{{ deploy_container.docker_registry_url }}" + name: "{{ deploy_container.container_name }}" + tag: "{{ deploy_container.container_tag | default('latest') }}" + push: true + source: build + delegate_to: localhost diff --git a/roles/deploy_container/templates/Dockerfile.j2 b/roles/deploy_container/templates/Dockerfile.j2 new file mode 100644 index 00000000..d0f5e81c --- /dev/null +++ b/roles/deploy_container/templates/Dockerfile.j2 @@ -0,0 +1,7 @@ +# Basic Dockerfile example +FROM debian:bullseye-slim +MAINTAINER sysadm@codeenigma.com + +RUN apt-get update +RUN apt-get install –y nginx +CMD ["echo","Image created"] \ No newline at end of file From 685d221ed91b722911f31764a1f423912b8c4561 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 22 Jun 2023 17:54:35 +0200 Subject: [PATCH 021/190] Ecs deployments pr devel (#301) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/tasks/main.yml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 4eb9d87a..37013df2 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -10,23 +10,27 @@ ansible.builtin.set_fact: _docker_registry_username: "{{ deploy_container.docker_registry_user }}" _docker_registry_password: "{{ deploy_container.docker_registry_pass }}" + delegate_to: localhost # Token valid for 12 hours - name: Fetch AWS ECR registry login token. ansible.builtin.command: - command: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.profile }}" + cmd: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.profile }}" when: deploy_container.aws_ecr.enabled + delegate_to: localhost register: _docker_registry_ecr_token - name: Set AWS ECR registry password. ansible.builtin.set_fact: _docker_registry_password: "{{ _docker_registry_ecr_token.stdout }}" when: deploy_container.aws_ecr.enabled + delegate_to: localhost - name: Set AWS ECR registry username. ansible.builtin.set_fact: _docker_registry_username: "AWS" when: deploy_container.aws_ecr.enabled + delegate_to: localhost - name: Log into Docker registry. community.docker.docker_login: @@ -35,7 +39,6 @@ password: "{{ _docker_registry_password }}" reauthorize: true delegate_to: localhost - when: deploy_container.docker_registry_login - name: Build and push container image. community.docker.docker_image: From e58461701adf4c4a21ab95e7e93440672e17f94d Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 22 Jun 2023 18:13:22 +0200 Subject: [PATCH 022/190] Ecs deployments pr devel (#302) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 12 +++++++++++- roles/deploy_container/README.md | 10 +++++++++- roles/deploy_container/defaults/main.yml | 1 + roles/deploy_container/tasks/main.yml | 2 +- .../templates/{Dockerfile.j2 => example.j2} | 2 +- 5 files changed, 23 insertions(+), 4 deletions(-) rename roles/deploy_container/templates/{Dockerfile.j2 => example.j2} (80%) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 46285580..e34d028c 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -1,5 +1,14 @@ # Deploy container -Step that deploys the codebase in a Docker container image. +Step that deploys the codebase in a Docker container image. Requires Docker and the `community.docker` collection for Ansible to be installed on your deploy server. You will also need to add a `docker` group and make sure your local deploy user is in that group, for example: + +``` +sudo groupadd docker +sudo usermod -aG docker deploy +``` + +This can be handled automatically by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. + +AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the managed AWS `EC2InstanceProfileForImageBuilderECRContainerBuilds` policy attached via IAM to allow access to fetch credentials and push containers. @@ -16,6 +25,7 @@ deploy_container: docker_registry_pass: asdf1234 docker_base_command: "docker image build" docker_build_dir: "{{ _ce_deploy_build_dir }}" + dockerfile_template: example.j2 # provide a templates directory next to your playbook and change this to match your Dockerfile template name environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached aws_ecr: diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 0647af50..e34d028c 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -1,5 +1,12 @@ # Deploy container -Step that deploys the codebase in a Docker container image. Requires Docker and the `community.docker` collection for Ansible to be installed on your deploy server. This can be handled by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. +Step that deploys the codebase in a Docker container image. Requires Docker and the `community.docker` collection for Ansible to be installed on your deploy server. You will also need to add a `docker` group and make sure your local deploy user is in that group, for example: + +``` +sudo groupadd docker +sudo usermod -aG docker deploy +``` + +This can be handled automatically by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the managed AWS `EC2InstanceProfileForImageBuilderECRContainerBuilds` policy attached via IAM to allow access to fetch credentials and push containers. @@ -18,6 +25,7 @@ deploy_container: docker_registry_pass: asdf1234 docker_base_command: "docker image build" docker_build_dir: "{{ _ce_deploy_build_dir }}" + dockerfile_template: example.j2 # provide a templates directory next to your playbook and change this to match your Dockerfile template name environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached aws_ecr: diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 4e4d30c3..8b1e8e69 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -7,6 +7,7 @@ deploy_container: docker_registry_pass: asdf1234 docker_base_command: "docker image build" docker_build_dir: "{{ _ce_deploy_build_dir }}" + dockerfile_template: example.j2 # provide a templates directory next to your playbook and change this to match your Dockerfile template name environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached aws_ecr: diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 37013df2..2b037a40 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -3,7 +3,7 @@ - name: Create Dockerfile from template. local_action: module: ansible.builtin.template - src: Dockerfile.j2 + src: "{{ deploy_container.dockerfile_template }}" dest: "{{ deploy_container.docker_build_dir }}/Dockerfile" - name: Set Docker registry username and password. diff --git a/roles/deploy_container/templates/Dockerfile.j2 b/roles/deploy_container/templates/example.j2 similarity index 80% rename from roles/deploy_container/templates/Dockerfile.j2 rename to roles/deploy_container/templates/example.j2 index d0f5e81c..5a155197 100644 --- a/roles/deploy_container/templates/Dockerfile.j2 +++ b/roles/deploy_container/templates/example.j2 @@ -3,5 +3,5 @@ FROM debian:bullseye-slim MAINTAINER sysadm@codeenigma.com RUN apt-get update -RUN apt-get install –y nginx +RUN apt-get install -y nginx CMD ["echo","Image created"] \ No newline at end of file From f1c89c03777a5758ef19ba393e912c8b8f9c5ced Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 22 Jun 2023 18:20:43 +0200 Subject: [PATCH 023/190] Ecs deployments pr devel (#303) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/defaults/main.yml | 1 + roles/deploy_container/tasks/main.yml | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 8b1e8e69..77fedec4 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -2,6 +2,7 @@ deploy_container: container_name: "example/example" container_tag: latest # tag will take format container_name:container_tag + docker_registry_name: index.docker.io docker_registry_url: https://index.docker.io/v1/ docker_registry_user: example docker_registry_pass: asdf1234 diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 2b037a40..f8984bcc 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -44,7 +44,7 @@ community.docker.docker_image: build: path: "{{ deploy_container.docker_build_dir }}" - repository: "{{ deploy_container.docker_registry_url }}" + repository: "{{ deploy_container.docker_registry_name }}" name: "{{ deploy_container.container_name }}" tag: "{{ deploy_container.container_tag | default('latest') }}" push: true From 36c7b118366ca31775b172ae94ab1d1cc13bddf9 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 22 Jun 2023 18:33:59 +0200 Subject: [PATCH 024/190] Ecs deployments pr devel (#304) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 4 ++-- roles/deploy_container/README.md | 4 ++-- roles/deploy_container/defaults/main.yml | 5 ++--- roles/deploy_container/tasks/main.yml | 3 +-- 4 files changed, 7 insertions(+), 9 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index e34d028c..9730b01e 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -18,9 +18,9 @@ AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the ```yaml --- deploy_container: - container_name: "example/example" + container_name: example container_tag: latest # tag will take format container_name:container_tag - docker_registry_url: https://index.docker.io/v1/ + docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name docker_registry_user: example docker_registry_pass: asdf1234 docker_base_command: "docker image build" diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index e34d028c..9730b01e 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -18,9 +18,9 @@ AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the ```yaml --- deploy_container: - container_name: "example/example" + container_name: example container_tag: latest # tag will take format container_name:container_tag - docker_registry_url: https://index.docker.io/v1/ + docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name docker_registry_user: example docker_registry_pass: asdf1234 docker_base_command: "docker image build" diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 77fedec4..3578e12a 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -1,9 +1,8 @@ --- deploy_container: - container_name: "example/example" + container_name: example container_tag: latest # tag will take format container_name:container_tag - docker_registry_name: index.docker.io - docker_registry_url: https://index.docker.io/v1/ + docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name docker_registry_user: example docker_registry_pass: asdf1234 docker_base_command: "docker image build" diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index f8984bcc..c58bb8b1 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -44,8 +44,7 @@ community.docker.docker_image: build: path: "{{ deploy_container.docker_build_dir }}" - repository: "{{ deploy_container.docker_registry_name }}" - name: "{{ deploy_container.container_name }}" + name: "{{ deploy_container.docker_registry_name }}/{{ deploy_container.container_name }}" tag: "{{ deploy_container.container_tag | default('latest') }}" push: true source: build From 2b36c1f5c15c261fbff410c62f3606707b5b873e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 12:47:20 +0200 Subject: [PATCH 025/190] Ecs deployments pr devel (#305) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 64 ++++- roles/deploy_container/README.md | 64 ++++- roles/deploy_container/defaults/main.yml | 64 ++++- roles/deploy_container/tasks/main.yml | 297 ++++++++++++++++++++++- roles/deploy_container/tasks/subnet.yml | 13 + 5 files changed, 495 insertions(+), 7 deletions(-) create mode 100644 roles/deploy_container/tasks/subnet.yml diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 9730b01e..0e4fe6fc 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -31,7 +31,69 @@ deploy_container: aws_ecr: enabled: false # set to true if using AWS ECR region: eu-west-1 - profile: example + aws_profile: example + # Requires the deploy IAM user to have the managed AmazonECS_FullAccess and ElasticLoadBalancingFullAccess policies attached + # Note, you can if you wish make more restrictive roles and policies + aws_ecs: + enabled: false + region: eu-west-1 + aws_profile: example + tags: {} + domain_name: www.example.com + route_53: + zone: example.com + aws_profile: example2 # might not be the same account + vpc_name: example + #vpc_id: vpc-XXXXXXX # optionally specify VPC ID to use + subnets: # list of public subnet names + - example-dev-a + - example-dev-b + security_groups: [] # list of security groups, accepts names or IDs + cluster_name: example + family_name: example + task_definition_revision: "" # integer, but must be presented as a string for Jinja2 + task_count: 1 + task_minimum_count: 1 + task_maximum_count: 4 + # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html + service_autoscale_metric_type: ECSServiceAverageCPUUtilization + service_autoscale_up_cooldown: 120 + service_autoscale_down_cooldown: 120 + service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_elb_container_name: example # the name of the container to be load balanced + execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable + containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + #cpu: 512 # these values can be set globally or per container + #memory: 1024 + launch_type: FARGATE + network_mode: awsvpc + #volumes: [] # list of additional volumes to attach + target_group_name: example # 32 character limit + target_group_protocol: http + target_group_port: 80 + targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service + #- Id: 10.0.0.2 + # Port: 80 + # AvailabilityZone: all + health_check: + protocol: http + path: / + response_codes: "200" + # Requires the deploy IAM user to have the managed AWSCertificateManagerFullAccess and AmazonRoute53FullAccess policies attached + acm: # see https://github.com/codeenigma/ce-provision/tree/1.x/roles/aws/aws_acm + create_cert: false + extra_domains: [] # list of Subject Alternative Name domains and zones + ssl_certificate_ARN: "" # optional SSL cert ARN if you imported one into AWS Certificate Manager + elb_security_groups: [] # default SG is used if none provided - module supports names or IDs + elb_http_port: 80 + elb_https_port: 443 + elb_ssl_policy: ELBSecurityPolicy-TLS13-1-2-2021-06 # see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#describe-ssl-policies + elb_listener_http_rules: [] + elb_listener_https_rules: [] + # Add custom listeners. See https://docs.ansible.com/ansible/latest/collections/amazon/aws/elb_application_lb_module.html + elb_listeners: [] + elb_idle_timeout: 60 + elb_ip_address_type: "ipv4" # Can be 'ipv4' or 'dualstack' (the latter includes IPv4 and IPv6 addresses). ``` diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 9730b01e..0e4fe6fc 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -31,7 +31,69 @@ deploy_container: aws_ecr: enabled: false # set to true if using AWS ECR region: eu-west-1 - profile: example + aws_profile: example + # Requires the deploy IAM user to have the managed AmazonECS_FullAccess and ElasticLoadBalancingFullAccess policies attached + # Note, you can if you wish make more restrictive roles and policies + aws_ecs: + enabled: false + region: eu-west-1 + aws_profile: example + tags: {} + domain_name: www.example.com + route_53: + zone: example.com + aws_profile: example2 # might not be the same account + vpc_name: example + #vpc_id: vpc-XXXXXXX # optionally specify VPC ID to use + subnets: # list of public subnet names + - example-dev-a + - example-dev-b + security_groups: [] # list of security groups, accepts names or IDs + cluster_name: example + family_name: example + task_definition_revision: "" # integer, but must be presented as a string for Jinja2 + task_count: 1 + task_minimum_count: 1 + task_maximum_count: 4 + # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html + service_autoscale_metric_type: ECSServiceAverageCPUUtilization + service_autoscale_up_cooldown: 120 + service_autoscale_down_cooldown: 120 + service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_elb_container_name: example # the name of the container to be load balanced + execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable + containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + #cpu: 512 # these values can be set globally or per container + #memory: 1024 + launch_type: FARGATE + network_mode: awsvpc + #volumes: [] # list of additional volumes to attach + target_group_name: example # 32 character limit + target_group_protocol: http + target_group_port: 80 + targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service + #- Id: 10.0.0.2 + # Port: 80 + # AvailabilityZone: all + health_check: + protocol: http + path: / + response_codes: "200" + # Requires the deploy IAM user to have the managed AWSCertificateManagerFullAccess and AmazonRoute53FullAccess policies attached + acm: # see https://github.com/codeenigma/ce-provision/tree/1.x/roles/aws/aws_acm + create_cert: false + extra_domains: [] # list of Subject Alternative Name domains and zones + ssl_certificate_ARN: "" # optional SSL cert ARN if you imported one into AWS Certificate Manager + elb_security_groups: [] # default SG is used if none provided - module supports names or IDs + elb_http_port: 80 + elb_https_port: 443 + elb_ssl_policy: ELBSecurityPolicy-TLS13-1-2-2021-06 # see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#describe-ssl-policies + elb_listener_http_rules: [] + elb_listener_https_rules: [] + # Add custom listeners. See https://docs.ansible.com/ansible/latest/collections/amazon/aws/elb_application_lb_module.html + elb_listeners: [] + elb_idle_timeout: 60 + elb_ip_address_type: "ipv4" # Can be 'ipv4' or 'dualstack' (the latter includes IPv4 and IPv6 addresses). ``` diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 3578e12a..d84b0d72 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -13,4 +13,66 @@ deploy_container: aws_ecr: enabled: false # set to true if using AWS ECR region: eu-west-1 - profile: example + aws_profile: example + # Requires the deploy IAM user to have the managed AmazonECS_FullAccess and ElasticLoadBalancingFullAccess policies attached + # Note, you can if you wish make more restrictive roles and policies + aws_ecs: + enabled: false + region: eu-west-1 + aws_profile: example + tags: {} + domain_name: www.example.com + route_53: + zone: example.com + aws_profile: example2 # might not be the same account + vpc_name: example + #vpc_id: vpc-XXXXXXX # optionally specify VPC ID to use + subnets: # list of public subnet names + - example-dev-a + - example-dev-b + security_groups: [] # list of security groups, accepts names or IDs + cluster_name: example + family_name: example + task_definition_revision: "" # integer, but must be presented as a string for Jinja2 + task_count: 1 + task_minimum_count: 1 + task_maximum_count: 4 + # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html + service_autoscale_metric_type: ECSServiceAverageCPUUtilization + service_autoscale_up_cooldown: 120 + service_autoscale_down_cooldown: 120 + service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_elb_container_name: example # the name of the container to be load balanced + execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable + containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + #cpu: 512 # these values can be set globally or per container + #memory: 1024 + launch_type: FARGATE + network_mode: awsvpc + #volumes: [] # list of additional volumes to attach + target_group_name: example # 32 character limit + target_group_protocol: http + target_group_port: 80 + targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service + #- Id: 10.0.0.2 + # Port: 80 + # AvailabilityZone: all + health_check: + protocol: http + path: / + response_codes: "200" + # Requires the deploy IAM user to have the managed AWSCertificateManagerFullAccess and AmazonRoute53FullAccess policies attached + acm: # see https://github.com/codeenigma/ce-provision/tree/1.x/roles/aws/aws_acm + create_cert: false + extra_domains: [] # list of Subject Alternative Name domains and zones + ssl_certificate_ARN: "" # optional SSL cert ARN if you imported one into AWS Certificate Manager + elb_security_groups: [] # default SG is used if none provided - module supports names or IDs + elb_http_port: 80 + elb_https_port: 443 + elb_ssl_policy: ELBSecurityPolicy-TLS13-1-2-2021-06 # see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#describe-ssl-policies + elb_listener_http_rules: [] + elb_listener_https_rules: [] + # Add custom listeners. See https://docs.ansible.com/ansible/latest/collections/amazon/aws/elb_application_lb_module.html + elb_listeners: [] + elb_idle_timeout: 60 + elb_ip_address_type: "ipv4" # Can be 'ipv4' or 'dualstack' (the latter includes IPv4 and IPv6 addresses). diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index c58bb8b1..dd64309f 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -1,5 +1,5 @@ --- -# @TODO - for AWS ECR we'll need certain policies attaching to the deploy IAM user +# Build and ship a container image - name: Create Dockerfile from template. local_action: module: ansible.builtin.template @@ -12,10 +12,9 @@ _docker_registry_password: "{{ deploy_container.docker_registry_pass }}" delegate_to: localhost -# Token valid for 12 hours -- name: Fetch AWS ECR registry login token. +- name: Fetch AWS ECR registry login token. # token valid for 12 hours ansible.builtin.command: - cmd: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.profile }}" + cmd: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.aws_profile }}" when: deploy_container.aws_ecr.enabled delegate_to: localhost register: _docker_registry_ecr_token @@ -49,3 +48,293 @@ push: true source: build delegate_to: localhost + +# Fetch the ACM role from ce-provision +- name: Ensure the aws_acm directory exists. + ansible.builtin.file: + path: "{{ _ce_deploy_base_dir }}/roles/aws_acm/{{ item }}" + state: directory + mode: '0755' + delegate_to: localhost + with_items: + - tasks + - defaults + +- name: Fetch the aws_acm files. + ansible.builtin.get_url: + url: "https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/{{ item }}/main.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/{{ item }}/main.yml" + delegate_to: localhost + with_items: + - tasks + - defaults + +- name: Fetch the aws_acm tasks. + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/tasks/main.yml + dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/tasks/main.yml" + delegate_to: localhost + +# Gather all network information +- name: Gather VPC information. + amazon.aws.ec2_vpc_net_info: + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + region: "{{ deploy_container.aws_ecs.region }}" + filters: + "tag:Name": "{{ deploy_container.aws_ecs.vpc_name }}" + register: _aws_ecs_cluster_vpc + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + - deploy_container.aws_ecs.vpc_name is defined + - deploy_container.aws_ecs.vpc_name | length > 0 + +- name: Set the VPC id from name. + ansible.builtin.set_fact: + _aws_ecs_cluster_vpc_id: "{{ _aws_ecs_cluster_vpc.vpcs[0].vpc_id }}" + when: + - deploy_container.aws_ecs.enabled + - deploy_container.aws_ecs.vpc_name is defined + - deploy_container.aws_ecs.vpc_name | length > 0 + +- name: Use provided VPC id. + ansible.builtin.set_fact: + _aws_ecs_cluster_vpc_id: "{{ deploy_container.aws_ecs.vpc_id }}" + when: + - deploy_container.aws_ecs.enabled + - (deploy_container.aws_ecs.vpc_name is not defined or deploy_container.aws_ecs.vpc_name | length < 0) + +- name: Reset subnets lists. + ansible.builtin.set_fact: + _aws_ecs_cluster_public_subnets_ids: [] + when: deploy_container.aws_ecs.enabled + +- name: Construct list of public subnet IDs. + ansible.builtin.include_tasks: subnet.yml + with_items: "{{ deploy_container.aws_ecs.subnets }}" + loop_control: + loop_var: subnet + when: deploy_container.aws_ecs.enabled + +# Construct AWS supporting assets +- name: Create task definition. + community.aws.ecs_taskdefinition: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + family: "{{ deploy_container.aws_ecs.family_name }}" + execution_role_arn: "{{ deploy_container.aws_ecs.execution_role_arn }}" + containers: "{{ deploy_container.aws_ecs.containers }}" + launch_type: "{{ deploy_container.aws_ecs.launch_type }}" + cpu: "{{ deploy_container.aws_ecs.cpu | default(omit) }}" + memory: "{{ deploy_container.aws_ecs.memory | default(omit) }}" + state: present + network_mode: "{{ deploy_container.aws_ecs.network_mode }}" + volumes: "{{ deploy_container.aws_ecs.volumes | default(omit) }}" + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Create a target group with IP address targets. + community.aws.elb_target_group: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit + protocol: "{{ deploy_container.aws_ecs.target_group_protocol }}" + port: "{{ deploy_container.aws_ecs.target_group_port }}" + vpc_id: "{{ _aws_ecs_cluster_vpc_id }}" + health_check_protocol: "{{ deploy_container.aws_ecs.health_check.protocol }}" + health_check_path: "{{ deploy_container.aws_ecs.health_check.path }}" + successful_response_codes: "{{ deploy_container.aws_ecs.health_check.response_codes }}" + target_type: ip + targets: "{{ deploy_container.aws_ecs.targets }}" + state: present + wait_timeout: 200 + wait: true + register: _aws_ecs_target_group + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Create SSL certificate for load balancer. + ansible.builtin.include_role: + name: aws_acm + vars: + aws_acm: + export: false + domain_name: "{{ deploy_container.aws_ecs.domain_name }}" + extra_domains: "{{ deploy_container.aws_ecs.acm.extra_domains }}" + route_53: + aws_profile: "{{ deploy_container.aws_ecs.route_53.aws_profile }}" + zone: "{{ deploy_container.aws_ecs.route_53.zone }}" + when: + - deploy_container.aws_ecs.acm.create_cert + - deploy_container.aws_ecs.enabled + +- name: Default to provided SSL certificate ARN. + ansible.builtin.set_fact: + _ssl_certificate_ARN: "{{ deploy_container.aws_ecs.ssl_certificate_ARN }}" + when: deploy_container.aws_ecs.enabled + +- name: If provided, override SSL certificate ARN with the one received from ACM. + ansible.builtin.set_fact: + _ssl_certificate_ARN: "{{ aws_acm_certificate_arn }}" + when: + - deploy_container.aws_ecs.acm.create_cert + - deploy_container.aws_ecs.enabled + +- name: Define default ALB listeners. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners_http: + Protocol: HTTP + Port: "{{ deploy_container.aws_ecs.elb_http_port }}" + DefaultActions: + - Type: forward + TargetGroupName: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" + Rules: "{{ deploy_container.aws_ecs.elb_listener_http_rules }}" + _aws_ecs_cluster_listeners_redirect: + Protocol: HTTP + Port: "{{ deploy_container.aws_ecs.elb_http_port }}" + DefaultActions: + - Type: redirect + RedirectConfig: + Protocol: HTTPS + Host: "#{host}" + Query: "#{query}" + Path: "/#{path}" + Port: "{{ deploy_container.aws_ecs.elb_https_port }}" + StatusCode: HTTP_301 + _aws_ecs_cluster_listeners_https: + Protocol: HTTPS + Port: "{{ deploy_container.aws_ecs.elb_https_port }}" + SslPolicy: "{{ deploy_container.aws_ecs.elb_ssl_policy }}" + Certificates: + - CertificateArn: "{{ _ssl_certificate_ARN }}" + DefaultActions: + - Type: forward + TargetGroupName: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" + Rules: "{{ deploy_container.aws_ecs.elb_listener_https_rules }}" + when: deploy_container.aws_ecs.enabled + +- name: Add HTTP listeners. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners: "{{ [ _aws_ecs_cluster_listeners_http ] }}" + when: + - _ssl_certificate_ARN | length < 1 + - deploy_container.aws_ecs.enabled + +- name: Add HTTPS Listener. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners: "{{ [ _aws_ecs_cluster_listeners_redirect, _aws_ecs_cluster_listeners_https ] }}" + when: + - _ssl_certificate_ARN | length > 1 + - deploy_container.aws_ecs.enabled + +- name: Add custom Listeners. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners: "{{ _aws_ecs_cluster_listeners + deploy_container.aws_ecs.elb_listeners }}" + when: + - deploy_container.aws_ecs.elb_listeners is defined + - deploy_container.aws_ecs.elb_listeners | length + - deploy_container.aws_ecs.enabled + +- name: Create an ALB. + amazon.aws.elb_application_lb: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit + state: present + tags: "{{ deploy_container.aws_ecs.tags }}" + subnets: "{{ _aws_ecs_cluster_public_subnets_ids }}" + security_groups: "{{ deploy_container.aws_ecs.elb_security_groups }}" + listeners: "{{ _aws_ecs_cluster_listeners }}" + idle_timeout: "{{ deploy_container.aws_ecs.elb_idle_timeout }}" + ip_address_type: "{{ aws_elb.ip_address_type }}" + register: _aws_ecs_cluster_alb + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Set task definition name. + ansible.builtin.set_fact: + _aws_ecs_service_task_definition: "{{ deploy_container.aws_ecs.family_name }}" + when: deploy_container.aws_ecs.enabled + +- name: Set task definition revision if applicable. + ansible.builtin.set_fact: + _aws_ecs_service_task_definition: "{{ deploy_container.aws_ecs.family_name }}:{{ deploy_container.aws_ecs.task_definition_revision }}" + when: + - deploy_container.aws_ecs.task_definition_revision | length > 0 + - deploy_container.aws_ecs.enabled + +- name: Create ECS service. + community.aws.ecs_service: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + state: present + name: "{{ deploy_container.aws_ecs.family_name }}" + cluster: "{{ deploy_container.aws_ecs.cluster_name }}" + task_definition: "{{ _aws_ecs_service_task_definition }}" + desired_count: "{{ deploy_container.aws_ecs.task_count }}" + launch_type: "{{ deploy_container.aws_ecs.launch_type }}" + platform_version: LATEST + load_balancers: # see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LoadBalancer.html + - containerName: "{{ deploy_container.aws_ecs.service_elb_container_name }}" + containerPort: "{{ deploy_container.aws_ecs.target_group_port }}" + targetGroupArn: "{{ _aws_ecs_target_group.target_group_arn }}" + network_configuration: + subnets: "{{ _aws_ecs_cluster_public_subnets_ids }}" + security_groups: "{{ deploy_container.aws_ecs.security_groups }}" + assign_public_ip: true # must be true for now - details: https://stackoverflow.com/a/66802973 + tags: "{{ deploy_container.aws_ecs.tags }}" + wait: true + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Create target tracking scaling policy for ECS service. + community.aws.application_autoscaling_policy: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + state: present + policy_name: "{{ deploy_container.aws_ecs.family_name }}" + service_namespace: ecs + resource_id: "service/{{ deploy_container.aws_ecs.cluster_name }}/{{ deploy_container.aws_ecs.family_name }}" + scalable_dimension: ecs:service:DesiredCount + minimum_tasks: "{{ deploy_container.aws_ecs.task_minimum_count }}" + maximum_tasks: "{{ deploy_container.aws_ecs.task_maximum_count }}" + policy_type: TargetTrackingScaling + target_tracking_scaling_policy_configuration: + PredefinedMetricSpecification: + PredefinedMetricType: "{{ deploy_container.aws_ecs.service_autoscale_metric_type }}" + ScaleInCooldown: "{{ deploy_container.aws_ecs.service_autoscale_up_cooldown }}" + ScaleOutCooldown: "{{ deploy_container.aws_ecs.service_autoscale_down_cooldown }}" + DisableScaleIn: false + TargetValue: "{{ deploy_container.aws_ecs.service_autoscale_target_value }}" + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Initialise the domains loop var with main domain entry DNS settings. + ansible.builtin.set_fact: + _aws_ecs_cluster_dns_all_domains: + - domain: "{{ deploy_container.aws_ecs.domain_name }}" + zone: "{{ deploy_container.aws_ecs.route_53.zone }}" + aws_profile: "{{ deploy_container.aws_ecs.route_53.aws_profile }}" + when: deploy_container.aws_ecs.enabled + +- name: Add extra_domains so we can loop through DNS records. + ansible.builtin.set_fact: + _aws_ecs_cluster_dns_all_domains: "{{ _aws_ecs_cluster_dns_all_domains + [{'domain': item.domain, 'zone': item.zone, 'aws_profile': item.aws_profile}] }}" + loop: "{{ deploy_container.aws_ecs.acm.extra_domains }}" + when: + - deploy_container.aws_ecs.acm.extra_domains | length > 0 + - deploy_container.aws_ecs.enabled + +- name: Add DNS records in Route 53. + amazon.aws.route53: + state: present + profile: "{{ item.aws_profile }}" + zone: "{{ item.zone }}" + record: "{{ item.domain }}" + type: CNAME + value: "{{ _aws_ecs_cluster_alb.dns_name }}" + overwrite: true + loop: "{{ _aws_ecs_cluster_dns_all_domains }}" + when: + - deploy_container.aws_ecs.route_53.zone | length > 0 + - deploy_container.aws_ecs.enabled \ No newline at end of file diff --git a/roles/deploy_container/tasks/subnet.yml b/roles/deploy_container/tasks/subnet.yml new file mode 100644 index 00000000..3cb3a06a --- /dev/null +++ b/roles/deploy_container/tasks/subnet.yml @@ -0,0 +1,13 @@ +- name: Gather public subnet information. + amazon.aws.ec2_vpc_subnet_info: + profile: "{{ deploy_container.aws_ecs.profile }}" + region: "{{ deploy_container.aws_ecs.region }}" + filters: + vpc-id: "{{ _aws_ecs_cluster_vpc_id }}" + tag:Name: "{{ subnet }}" + register: _aws_ecs_cluster_public_subnet + delegate_to: localhost + +- name: Add public subnet to the list. + ansible.builtin.set_fact: + _aws_ecs_cluster_public_subnets_ids: "{{ _aws_ecs_cluster_public_subnets_ids + [ _aws_ecs_cluster_public_subnet.subnets[0].subnet_id ] }}" From cda04d4d55e77803891771b56d7d80656fcc1732 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 13:14:43 +0200 Subject: [PATCH 026/190] Ecs deployments pr devel (#306) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/defaults/main.yml | 1 - roles/deploy_container/tasks/main.yml | 6 +++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index d84b0d72..cee7ce12 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -42,7 +42,6 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at - service_elb_container_name: example # the name of the container to be load balanced execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers #cpu: 512 # these values can be set globally or per container diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index dd64309f..17f339be 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -31,6 +31,10 @@ when: deploy_container.aws_ecr.enabled delegate_to: localhost +- name: Log out of Docker registry to force a login. + community.docker.docker_login: + state: absent + - name: Log into Docker registry. community.docker.docker_login: registry_url: "{{ deploy_container.docker_registry_url }}" @@ -275,7 +279,7 @@ launch_type: "{{ deploy_container.aws_ecs.launch_type }}" platform_version: LATEST load_balancers: # see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LoadBalancer.html - - containerName: "{{ deploy_container.aws_ecs.service_elb_container_name }}" + - containerName: "{{ deploy_container.container_name }}" containerPort: "{{ deploy_container.aws_ecs.target_group_port }}" targetGroupArn: "{{ _aws_ecs_target_group.target_group_arn }}" network_configuration: From 401f2eea0c8a8461d2276a3f8baaf4a540495d01 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 13:26:53 +0200 Subject: [PATCH 027/190] Ecs deployments pr devel (#307) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 1 - roles/deploy_container/README.md | 1 - roles/deploy_container/tasks/main.yml | 1 + 3 files changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 0e4fe6fc..10292af6 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -60,7 +60,6 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at - service_elb_container_name: example # the name of the container to be load balanced execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers #cpu: 512 # these values can be set globally or per container diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 0e4fe6fc..10292af6 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -60,7 +60,6 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at - service_elb_container_name: example # the name of the container to be load balanced execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers #cpu: 512 # these values can be set globally or per container diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 17f339be..05e1ffc0 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -34,6 +34,7 @@ - name: Log out of Docker registry to force a login. community.docker.docker_login: state: absent + delegate_to: localhost - name: Log into Docker registry. community.docker.docker_login: From 5542af7408af45c4bb9d60a6e74d9849b5a2998e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 13:31:35 +0200 Subject: [PATCH 028/190] Ecs deployments pr devel (#308) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/tasks/main.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 05e1ffc0..83cb959a 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -31,9 +31,10 @@ when: deploy_container.aws_ecr.enabled delegate_to: localhost -- name: Log out of Docker registry to force a login. - community.docker.docker_login: +- name: Remove Docker credentials file. + ansible.builtin.file: state: absent + path: "/home/{{ deploy_user }}/.docker/config.json" delegate_to: localhost - name: Log into Docker registry. From a7fcb6dfbf2f5c58b3bedcd6f93ed82677328565 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 13:38:46 +0200 Subject: [PATCH 029/190] Ecs deployments pr devel (#309) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/tasks/subnet.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_container/tasks/subnet.yml b/roles/deploy_container/tasks/subnet.yml index 3cb3a06a..a0701a36 100644 --- a/roles/deploy_container/tasks/subnet.yml +++ b/roles/deploy_container/tasks/subnet.yml @@ -1,6 +1,6 @@ - name: Gather public subnet information. amazon.aws.ec2_vpc_subnet_info: - profile: "{{ deploy_container.aws_ecs.profile }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" region: "{{ deploy_container.aws_ecs.region }}" filters: vpc-id: "{{ _aws_ecs_cluster_vpc_id }}" From 29a3cb7fd59c5328198e8d868cb78cdfb7e1118c Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 13:51:47 +0200 Subject: [PATCH 030/190] Ecs deployments pr devel (#310) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/tasks/main.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 83cb959a..af1164eb 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -164,6 +164,9 @@ name: aws_acm vars: aws_acm: + region: "{{ deploy_container.aws_ecs.region }}" + aws_profile: "{{ deploy_container.aws_ecs.aws_profile }}" + tags: "{{ deploy_container.aws_ecs.tags }}" export: false domain_name: "{{ deploy_container.aws_ecs.domain_name }}" extra_domains: "{{ deploy_container.aws_ecs.acm.extra_domains }}" From 1cea0ff7923c33f43b0dc289c9be7b496bce7f14 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 16:35:41 +0200 Subject: [PATCH 031/190] Ecs deployments pr devel (#311) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 1 + roles/deploy_container/README.md | 1 + roles/deploy_container/defaults/main.yml | 1 + roles/deploy_container/tasks/main.yml | 2 +- 4 files changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 10292af6..1ebaa80f 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -70,6 +70,7 @@ deploy_container: target_group_name: example # 32 character limit target_group_protocol: http target_group_port: 80 + target_group_wait_timeout: 200 # how long to wait for target group events to complete targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service #- Id: 10.0.0.2 # Port: 80 diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 10292af6..1ebaa80f 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -70,6 +70,7 @@ deploy_container: target_group_name: example # 32 character limit target_group_protocol: http target_group_port: 80 + target_group_wait_timeout: 200 # how long to wait for target group events to complete targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service #- Id: 10.0.0.2 # Port: 80 diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index cee7ce12..00195f6a 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -52,6 +52,7 @@ deploy_container: target_group_name: example # 32 character limit target_group_protocol: http target_group_port: 80 + target_group_wait_timeout: 200 # how long to wait for target group events to complete targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service #- Id: 10.0.0.2 # Port: 80 diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index af1164eb..053e7ab9 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -153,7 +153,7 @@ target_type: ip targets: "{{ deploy_container.aws_ecs.targets }}" state: present - wait_timeout: 200 + wait_timeout: "{{ deploy_container.aws_ecs.target_group_wait_timeout }}" wait: true register: _aws_ecs_target_group delegate_to: localhost From bc2117e491d4695f3e97e4f194f8063a4868400c Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 16:43:09 +0200 Subject: [PATCH 032/190] Ecs deployments pr devel (#312) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 4 ++-- roles/deploy_container/README.md | 4 ++-- roles/deploy_container/defaults/main.yml | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 1ebaa80f..d1befb03 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -62,8 +62,8 @@ deploy_container: service_autoscale_target_value: 70 # the value to trigger a scaling event at execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - #cpu: 512 # these values can be set globally or per container - #memory: 1024 + cpu: 512 # these values can be set globally or per container + memory: 1024 launch_type: FARGATE network_mode: awsvpc #volumes: [] # list of additional volumes to attach diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 1ebaa80f..d1befb03 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -62,8 +62,8 @@ deploy_container: service_autoscale_target_value: 70 # the value to trigger a scaling event at execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - #cpu: 512 # these values can be set globally or per container - #memory: 1024 + cpu: 512 # these values can be set globally or per container + memory: 1024 launch_type: FARGATE network_mode: awsvpc #volumes: [] # list of additional volumes to attach diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 00195f6a..22380049 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -44,8 +44,8 @@ deploy_container: service_autoscale_target_value: 70 # the value to trigger a scaling event at execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - #cpu: 512 # these values can be set globally or per container - #memory: 1024 + cpu: 512 # these values can be set globally or per container + memory: 1024 launch_type: FARGATE network_mode: awsvpc #volumes: [] # list of additional volumes to attach From 6c628626388baa45133fdf0344fce613e8d69b56 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 16:53:35 +0200 Subject: [PATCH 033/190] Ecs deployments pr devel (#313) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/tasks/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 053e7ab9..0fafdfc2 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -255,7 +255,7 @@ security_groups: "{{ deploy_container.aws_ecs.elb_security_groups }}" listeners: "{{ _aws_ecs_cluster_listeners }}" idle_timeout: "{{ deploy_container.aws_ecs.elb_idle_timeout }}" - ip_address_type: "{{ aws_elb.ip_address_type }}" + ip_address_type: "{{ deploy_container.aws_ecs.elb_ip_address_type }}" register: _aws_ecs_cluster_alb delegate_to: localhost when: deploy_container.aws_ecs.enabled From f5cbbfbd43e5aeafda19826003ffd1ffd886b146 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 18:15:20 +0200 Subject: [PATCH 034/190] Ecs deployments pr devel (#315) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/defaults/main.yml | 2 +- roles/deploy_container/tasks/main.yml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 22380049..77be7a6a 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -8,7 +8,7 @@ deploy_container: docker_base_command: "docker image build" docker_build_dir: "{{ _ce_deploy_build_dir }}" dockerfile_template: example.j2 # provide a templates directory next to your playbook and change this to match your Dockerfile template name - environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template + environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached aws_ecr: enabled: false # set to true if using AWS ECR diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 0fafdfc2..23f4d996 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -1,10 +1,10 @@ --- # Build and ship a container image - name: Create Dockerfile from template. - local_action: - module: ansible.builtin.template + ansible.builtin.template: src: "{{ deploy_container.dockerfile_template }}" dest: "{{ deploy_container.docker_build_dir }}/Dockerfile" + delegate_to: localhost - name: Set Docker registry username and password. ansible.builtin.set_fact: From c47f76a3285cdaf169a607049f2da53183cc732a Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 18:25:27 +0200 Subject: [PATCH 035/190] Ecs deployments pr devel (#316) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 3 ++- roles/deploy_container/README.md | 3 ++- roles/deploy_container/defaults/main.yml | 1 + roles/deploy_container/tasks/main.yml | 2 ++ 4 files changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index d1befb03..8f7dca28 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -20,13 +20,14 @@ AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the deploy_container: container_name: example container_tag: latest # tag will take format container_name:container_tag + container_force_build: true # force Docker to build and tag a new image docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name docker_registry_user: example docker_registry_pass: asdf1234 docker_base_command: "docker image build" docker_build_dir: "{{ _ce_deploy_build_dir }}" dockerfile_template: example.j2 # provide a templates directory next to your playbook and change this to match your Dockerfile template name - environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template + environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached aws_ecr: enabled: false # set to true if using AWS ECR diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index d1befb03..8f7dca28 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -20,13 +20,14 @@ AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the deploy_container: container_name: example container_tag: latest # tag will take format container_name:container_tag + container_force_build: true # force Docker to build and tag a new image docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name docker_registry_user: example docker_registry_pass: asdf1234 docker_base_command: "docker image build" docker_build_dir: "{{ _ce_deploy_build_dir }}" dockerfile_template: example.j2 # provide a templates directory next to your playbook and change this to match your Dockerfile template name - environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template + environment_vars: {} # dictionary you can populate for use in a custom Dockerfile template # Requires the deploy IAM user to have the managed EC2InstanceProfileForImageBuilderECRContainerBuilds policy attached aws_ecr: enabled: false # set to true if using AWS ECR diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 77be7a6a..7605f796 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -2,6 +2,7 @@ deploy_container: container_name: example container_tag: latest # tag will take format container_name:container_tag + container_force_build: true # force Docker to build and tag a new image docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name docker_registry_user: example docker_registry_pass: asdf1234 diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 23f4d996..34866aa1 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -53,6 +53,8 @@ tag: "{{ deploy_container.container_tag | default('latest') }}" push: true source: build + force_source: "{{ deploy_container.container_force_build }}" + force_tag: "{{ deploy_container.container_force_build }}" delegate_to: localhost # Fetch the ACM role from ce-provision From 9a10a050a46fb1d476c7f24cc249efb8c700c0a1 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 18:43:34 +0200 Subject: [PATCH 036/190] Catching up devel (#318) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> From 3a94844d61e5eab6d844e5e0cecb4db5d4fb4b1f Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 27 Jun 2023 19:53:18 +0300 Subject: [PATCH 037/190] Fix typo pr devel (#319) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo * fix_typo --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- roles/cache_clear/cache_clear-drupal8/tasks/main.yml | 1 - roles/database_apply/database_apply-drupal8/tasks/main.yml | 1 - roles/sync/database_sync/database_sync-mysql/tasks/sync.yml | 4 ++-- 3 files changed, 2 insertions(+), 4 deletions(-) diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index e9524b81..5289f254 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -15,4 +15,3 @@ ansible.builtin.debug: msg: "{{ _drush_output }}" when: drupal.drush_verbose_output - diff --git a/roles/database_apply/database_apply-drupal8/tasks/main.yml b/roles/database_apply/database_apply-drupal8/tasks/main.yml index e2fecd17..c423fad9 100644 --- a/roles/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/database_apply/database_apply-drupal8/tasks/main.yml @@ -85,4 +85,3 @@ ansible.builtin.debug: msg: "{{ _drush_output }}" when: drupal.drush_verbose_output - diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index ac346444..7e52bd07 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -40,14 +40,14 @@ ansible.builtin.set_fact: archive_file_type: "bz2" archival_command: "bzip2" - when: + when: - mysql_sync.archival_method == 'bzip2' - name: Register gunzip archive type vars. ansible.builtin.set_fact: archive_file_type: "gz" archival_command: "gzip" - when: + when: - mysql_sync.archival_method == 'gzip' - name: Register remote dump name (from database). From f33baadf0fdf7413fa96e44c03ceeae27c5b5792 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 27 Jun 2023 19:10:34 +0200 Subject: [PATCH 038/190] Ecs deployments pr devel (#321) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 22 ++++++++++++++++++- .../sync/database_sync/database_sync-mysql.md | 5 +++-- roles/deploy_container/README.md | 22 ++++++++++++++++++- .../database_sync-mysql/README.md | 5 +++-- 4 files changed, 48 insertions(+), 6 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 8f7dca28..3e5a9c30 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -8,7 +8,27 @@ sudo usermod -aG docker deploy This can be handled automatically by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. -AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the managed AWS `EC2InstanceProfileForImageBuilderECRContainerBuilds` policy attached via IAM to allow access to fetch credentials and push containers. +## AWS IAM requirements +AWS integration requires the AWS CLI user provided for `ce-deploy` to have certain managed AWS policies attached. + +If you enable AWS ECR registry integration by setting `deploy_container.aws_ecr.enabled` to `true` then you will need the `EC2InstanceProfileForImageBuilderECRContainerBuilds` policy attached via IAM to allow access to fetch credentials and push containers. + +Similarly, if you set `deploy_container.aws_ecs.acm.create_cert` to `true` then you will need the `AWSCertificateManagerFullAccess` policy attaching to create SSL certificates. + +If you enable full AWS ECS integration by setting `deploy_container.aws_ecs.enabled` to `true` then this requires the following policies to be attached to the AWS CLI user: +* `AmazonECS_FullAccess` - to create task definitions and services +* `ElasticLoadBalancingFullAccess` - to create load balancers and target groups + +Finally, if you set `deploy_container.aws_ecs.route_53.zone` to another other than an empty string then you will also need `AmazonRoute53FullAccess` attaching to manipulate DNS entries in Route 53. + +The full list is: +* `EC2InstanceProfileForImageBuilderECRContainerBuilds` - to manipulate images in AWS ECR +* `AWSCertificateManagerFullAccess` - to manage SSL certificates +* `AmazonECS_FullAccess` - to create task definitions and services +* `ElasticLoadBalancingFullAccess` - to create load balancers and target groups +* `AmazonRoute53FullAccess` - to manage DNS entries + +Naturally you can always create custom policies and roles to have tighter access control. This document simply gives you the broad strokes AWS managed policies you can use in conjunction with this Ansible role. diff --git a/docs/roles/sync/database_sync/database_sync-mysql.md b/docs/roles/sync/database_sync/database_sync-mysql.md index cbe1a353..d70d770e 100644 --- a/docs/roles/sync/database_sync/database_sync-mysql.md +++ b/docs/roles/sync/database_sync/database_sync-mysql.md @@ -5,8 +5,9 @@ Sync MySQL databases between environments. ```yaml --- mysql_sync: - mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here - cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes + mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here. + cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes. + archival_method: "bzip2" # oprions are "bzip2" or "gzip". databases: - source: # Name of the database to take a dump from. diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 8f7dca28..3e5a9c30 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -8,7 +8,27 @@ sudo usermod -aG docker deploy This can be handled automatically by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. -AWS ECR registries require the AWS CLI user provided for `ce-deploy` to have the managed AWS `EC2InstanceProfileForImageBuilderECRContainerBuilds` policy attached via IAM to allow access to fetch credentials and push containers. +## AWS IAM requirements +AWS integration requires the AWS CLI user provided for `ce-deploy` to have certain managed AWS policies attached. + +If you enable AWS ECR registry integration by setting `deploy_container.aws_ecr.enabled` to `true` then you will need the `EC2InstanceProfileForImageBuilderECRContainerBuilds` policy attached via IAM to allow access to fetch credentials and push containers. + +Similarly, if you set `deploy_container.aws_ecs.acm.create_cert` to `true` then you will need the `AWSCertificateManagerFullAccess` policy attaching to create SSL certificates. + +If you enable full AWS ECS integration by setting `deploy_container.aws_ecs.enabled` to `true` then this requires the following policies to be attached to the AWS CLI user: +* `AmazonECS_FullAccess` - to create task definitions and services +* `ElasticLoadBalancingFullAccess` - to create load balancers and target groups + +Finally, if you set `deploy_container.aws_ecs.route_53.zone` to another other than an empty string then you will also need `AmazonRoute53FullAccess` attaching to manipulate DNS entries in Route 53. + +The full list is: +* `EC2InstanceProfileForImageBuilderECRContainerBuilds` - to manipulate images in AWS ECR +* `AWSCertificateManagerFullAccess` - to manage SSL certificates +* `AmazonECS_FullAccess` - to create task definitions and services +* `ElasticLoadBalancingFullAccess` - to create load balancers and target groups +* `AmazonRoute53FullAccess` - to manage DNS entries + +Naturally you can always create custom policies and roles to have tighter access control. This document simply gives you the broad strokes AWS managed policies you can use in conjunction with this Ansible role. diff --git a/roles/sync/database_sync/database_sync-mysql/README.md b/roles/sync/database_sync/database_sync-mysql/README.md index cbe1a353..d70d770e 100644 --- a/roles/sync/database_sync/database_sync-mysql/README.md +++ b/roles/sync/database_sync/database_sync-mysql/README.md @@ -5,8 +5,9 @@ Sync MySQL databases between environments. ```yaml --- mysql_sync: - mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here - cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes + mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here. + cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes. + archival_method: "bzip2" # oprions are "bzip2" or "gzip". databases: - source: # Name of the database to take a dump from. From 71b0bc1c6aea134b8cd02548f08e2bc3687e682b Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 29 Jun 2023 14:33:21 +0200 Subject: [PATCH 039/190] Ecs deployments pr devel (#323) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 22 +++++++++++++++++----- roles/deploy_container/README.md | 22 +++++++++++++++++----- roles/deploy_container/defaults/main.yml | 22 +++++++++++++++++----- roles/deploy_container/tasks/main.yml | 1 + 4 files changed, 52 insertions(+), 15 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 3e5a9c30..41a87e88 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -38,7 +38,7 @@ Naturally you can always create custom policies and roles to have tighter access ```yaml --- deploy_container: - container_name: example + container_name: example-container container_tag: latest # tag will take format container_name:container_tag container_force_build: true # force Docker to build and tag a new image docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name @@ -70,8 +70,8 @@ deploy_container: - example-dev-a - example-dev-b security_groups: [] # list of security groups, accepts names or IDs - cluster_name: example - family_name: example + cluster_name: example-cluster + family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 task_count: 1 task_minimum_count: 1 @@ -82,7 +82,19 @@ deploy_container: service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable - containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + - name: example-container + essential: true + image: index.docker.io/example:latest + portMappings: + - containerPort: 8080 # should match target_group_port + hostPort: 8080 + logConfiguration: + logDriver: awslogs + options: + awslogs-group: /ecs/example-cluster + awslogs-region: eu-west-1 + awslogs-stream-prefix: "ecs-example-task" cpu: 512 # these values can be set globally or per container memory: 1024 launch_type: FARGATE @@ -90,7 +102,7 @@ deploy_container: #volumes: [] # list of additional volumes to attach target_group_name: example # 32 character limit target_group_protocol: http - target_group_port: 80 + target_group_port: 8080 # ports lower than 1024 will require the app to be configured to run as a privileged user in the Dockerfile target_group_wait_timeout: 200 # how long to wait for target group events to complete targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service #- Id: 10.0.0.2 diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 3e5a9c30..41a87e88 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -38,7 +38,7 @@ Naturally you can always create custom policies and roles to have tighter access ```yaml --- deploy_container: - container_name: example + container_name: example-container container_tag: latest # tag will take format container_name:container_tag container_force_build: true # force Docker to build and tag a new image docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name @@ -70,8 +70,8 @@ deploy_container: - example-dev-a - example-dev-b security_groups: [] # list of security groups, accepts names or IDs - cluster_name: example - family_name: example + cluster_name: example-cluster + family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 task_count: 1 task_minimum_count: 1 @@ -82,7 +82,19 @@ deploy_container: service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable - containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + - name: example-container + essential: true + image: index.docker.io/example:latest + portMappings: + - containerPort: 8080 # should match target_group_port + hostPort: 8080 + logConfiguration: + logDriver: awslogs + options: + awslogs-group: /ecs/example-cluster + awslogs-region: eu-west-1 + awslogs-stream-prefix: "ecs-example-task" cpu: 512 # these values can be set globally or per container memory: 1024 launch_type: FARGATE @@ -90,7 +102,7 @@ deploy_container: #volumes: [] # list of additional volumes to attach target_group_name: example # 32 character limit target_group_protocol: http - target_group_port: 80 + target_group_port: 8080 # ports lower than 1024 will require the app to be configured to run as a privileged user in the Dockerfile target_group_wait_timeout: 200 # how long to wait for target group events to complete targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service #- Id: 10.0.0.2 diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 7605f796..c660243a 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -1,6 +1,6 @@ --- deploy_container: - container_name: example + container_name: example-container container_tag: latest # tag will take format container_name:container_tag container_force_build: true # force Docker to build and tag a new image docker_registry_name: index.docker.io/example # combines with container_name to make the full registry name, docker_registry_name/container_name @@ -32,8 +32,8 @@ deploy_container: - example-dev-a - example-dev-b security_groups: [] # list of security groups, accepts names or IDs - cluster_name: example - family_name: example + cluster_name: example-cluster + family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 task_count: 1 task_minimum_count: 1 @@ -44,7 +44,19 @@ deploy_container: service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable - containers: [] # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers + - name: example-container + essential: true + image: index.docker.io/example:latest + portMappings: + - containerPort: 8080 # should match target_group_port + hostPort: 8080 + logConfiguration: + logDriver: awslogs + options: + awslogs-group: /ecs/example-cluster + awslogs-region: eu-west-1 + awslogs-stream-prefix: "ecs-example-task" cpu: 512 # these values can be set globally or per container memory: 1024 launch_type: FARGATE @@ -52,7 +64,7 @@ deploy_container: #volumes: [] # list of additional volumes to attach target_group_name: example # 32 character limit target_group_protocol: http - target_group_port: 80 + target_group_port: 8080 # ports lower than 1024 will require the app to be configured to run as a privileged user in the Dockerfile target_group_wait_timeout: 200 # how long to wait for target group events to complete targets: [] # typically we do not specify targets at this point, this will be handled automatically by the ECS service #- Id: 10.0.0.2 diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 34866aa1..17ef63cf 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -55,6 +55,7 @@ source: build force_source: "{{ deploy_container.container_force_build }}" force_tag: "{{ deploy_container.container_force_build }}" + force: "{{ deploy_container.container_force_build }}" delegate_to: localhost # Fetch the ACM role from ce-provision From d2cea3a971277c90fac00af3b49ad1a69269da3c Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 29 Jun 2023 14:39:30 +0200 Subject: [PATCH 040/190] Ecs deployments pr devel (#326) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_container/tasks/main.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 17ef63cf..34866aa1 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -55,7 +55,6 @@ source: build force_source: "{{ deploy_container.container_force_build }}" force_tag: "{{ deploy_container.container_force_build }}" - force: "{{ deploy_container.container_force_build }}" delegate_to: localhost # Fetch the ACM role from ce-provision From 184a9be42cca8b5829096b5981a4eea14572169e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 29 Jun 2023 14:55:51 +0200 Subject: [PATCH 041/190] Ecs deployments pr devel (#327) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 2 ++ roles/deploy_container/README.md | 2 ++ roles/deploy_container/defaults/main.yml | 2 ++ roles/deploy_container/tasks/main.yml | 2 ++ 4 files changed, 8 insertions(+) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 41a87e88..8e5872cb 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -73,6 +73,7 @@ deploy_container: cluster_name: example-cluster family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 + task_definition_force_create: false # creates a task definition revision every time if set to true task_count: 1 task_minimum_count: 1 task_maximum_count: 4 @@ -81,6 +82,7 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_force_refresh: false # forces a refresh of all containers if set to true execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - name: example-container diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 41a87e88..8e5872cb 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -73,6 +73,7 @@ deploy_container: cluster_name: example-cluster family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 + task_definition_force_create: false # creates a task definition revision every time if set to true task_count: 1 task_minimum_count: 1 task_maximum_count: 4 @@ -81,6 +82,7 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_force_refresh: false # forces a refresh of all containers if set to true execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - name: example-container diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index c660243a..0ca73b42 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -35,6 +35,7 @@ deploy_container: cluster_name: example-cluster family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 + task_definition_force_create: false # creates a task definition revision every time if set to true task_count: 1 task_minimum_count: 1 task_maximum_count: 4 @@ -43,6 +44,7 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_force_refresh: false # forces a refresh of all containers if set to true execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - name: example-container diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 34866aa1..8238e080 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -138,6 +138,7 @@ state: present network_mode: "{{ deploy_container.aws_ecs.network_mode }}" volumes: "{{ deploy_container.aws_ecs.volumes | default(omit) }}" + force_create: "{{ deploy_container.aws_ecs.task_definition_force_create }}" delegate_to: localhost when: deploy_container.aws_ecs.enabled @@ -294,6 +295,7 @@ security_groups: "{{ deploy_container.aws_ecs.security_groups }}" assign_public_ip: true # must be true for now - details: https://stackoverflow.com/a/66802973 tags: "{{ deploy_container.aws_ecs.tags }}" + force_new_deployment: "{{ deploy_container.aws_ecs.service_force_refresh }}" wait: true delegate_to: localhost when: deploy_container.aws_ecs.enabled From 006db4648ff4d53a393d769163efc694018263a1 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 29 Jun 2023 18:52:28 +0200 Subject: [PATCH 042/190] Ecs deployments pr devel (#328) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 2 ++ roles/deploy_container/README.md | 2 ++ roles/deploy_container/defaults/main.yml | 2 ++ roles/deploy_container/tasks/main.yml | 3 ++- 4 files changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 8e5872cb..9970dc39 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -82,6 +82,8 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 + service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API service_force_refresh: false # forces a refresh of all containers if set to true execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 8e5872cb..9970dc39 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -82,6 +82,8 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 + service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API service_force_refresh: false # forces a refresh of all containers if set to true execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 0ca73b42..a9b7e9d9 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -44,6 +44,8 @@ deploy_container: service_autoscale_up_cooldown: 120 service_autoscale_down_cooldown: 120 service_autoscale_target_value: 70 # the value to trigger a scaling event at + service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 + service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API service_force_refresh: false # forces a refresh of all containers if set to true execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 8238e080..de12e5bc 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -293,8 +293,9 @@ network_configuration: subnets: "{{ _aws_ecs_cluster_public_subnets_ids }}" security_groups: "{{ deploy_container.aws_ecs.security_groups }}" - assign_public_ip: true # must be true for now - details: https://stackoverflow.com/a/66802973 + assign_public_ip: "{{ deploy_container.aws_ecs.service_public_container_ip }}" tags: "{{ deploy_container.aws_ecs.tags }}" + enable_execute_command: "{{ deploy_container.aws_ecs.service_enable_ssm }}" force_new_deployment: "{{ deploy_container.aws_ecs.service_force_refresh }}" wait: true delegate_to: localhost From 76fbe18ba7690c77ee86bfcfad0c86b1bc1e4474 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 30 Jun 2023 09:59:01 +0200 Subject: [PATCH 043/190] Ecs deployments pr devel (#330) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 13 +++++++++---- roles/deploy_container/README.md | 13 +++++++++---- roles/deploy_container/defaults/main.yml | 13 +++++++++---- roles/deploy_container/tasks/main.yml | 17 +++++++++++++---- roles/deploy_container/tasks/subnet-private.yml | 13 +++++++++++++ .../tasks/{subnet.yml => subnet-public.yml} | 0 6 files changed, 53 insertions(+), 16 deletions(-) create mode 100644 roles/deploy_container/tasks/subnet-private.yml rename roles/deploy_container/tasks/{subnet.yml => subnet-public.yml} (100%) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 9970dc39..8152cedc 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -66,17 +66,20 @@ deploy_container: aws_profile: example2 # might not be the same account vpc_name: example #vpc_id: vpc-XXXXXXX # optionally specify VPC ID to use - subnets: # list of public subnet names - - example-dev-a - - example-dev-b security_groups: [] # list of security groups, accepts names or IDs cluster_name: example-cluster family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 task_definition_force_create: false # creates a task definition revision every time if set to true + task_execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable + #task_role_arn: "" # required if you set service_enable_ssm to true task_count: 1 task_minimum_count: 1 task_maximum_count: 4 + # These subnets are usually the subnets created by ce-provision when you made your ECS cluster and must have a NAT gateway for ECR access. + service_subnets: # list of private subnet names + - example-cluster-dev-a + - example-cluster-dev-b # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html service_autoscale_metric_type: ECSServiceAverageCPUUtilization service_autoscale_up_cooldown: 120 @@ -85,7 +88,6 @@ deploy_container: service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API service_force_refresh: false # forces a refresh of all containers if set to true - execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - name: example-container essential: true @@ -122,6 +124,9 @@ deploy_container: extra_domains: [] # list of Subject Alternative Name domains and zones ssl_certificate_ARN: "" # optional SSL cert ARN if you imported one into AWS Certificate Manager elb_security_groups: [] # default SG is used if none provided - module supports names or IDs + elb_subnets: # must be public subnets for public facing applications + - example-dev-a + - example-dev-b elb_http_port: 80 elb_https_port: 443 elb_ssl_policy: ELBSecurityPolicy-TLS13-1-2-2021-06 # see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#describe-ssl-policies diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 9970dc39..8152cedc 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -66,17 +66,20 @@ deploy_container: aws_profile: example2 # might not be the same account vpc_name: example #vpc_id: vpc-XXXXXXX # optionally specify VPC ID to use - subnets: # list of public subnet names - - example-dev-a - - example-dev-b security_groups: [] # list of security groups, accepts names or IDs cluster_name: example-cluster family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 task_definition_force_create: false # creates a task definition revision every time if set to true + task_execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable + #task_role_arn: "" # required if you set service_enable_ssm to true task_count: 1 task_minimum_count: 1 task_maximum_count: 4 + # These subnets are usually the subnets created by ce-provision when you made your ECS cluster and must have a NAT gateway for ECR access. + service_subnets: # list of private subnet names + - example-cluster-dev-a + - example-cluster-dev-b # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html service_autoscale_metric_type: ECSServiceAverageCPUUtilization service_autoscale_up_cooldown: 120 @@ -85,7 +88,6 @@ deploy_container: service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API service_force_refresh: false # forces a refresh of all containers if set to true - execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - name: example-container essential: true @@ -122,6 +124,9 @@ deploy_container: extra_domains: [] # list of Subject Alternative Name domains and zones ssl_certificate_ARN: "" # optional SSL cert ARN if you imported one into AWS Certificate Manager elb_security_groups: [] # default SG is used if none provided - module supports names or IDs + elb_subnets: # must be public subnets for public facing applications + - example-dev-a + - example-dev-b elb_http_port: 80 elb_https_port: 443 elb_ssl_policy: ELBSecurityPolicy-TLS13-1-2-2021-06 # see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#describe-ssl-policies diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index a9b7e9d9..be178aca 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -28,17 +28,20 @@ deploy_container: aws_profile: example2 # might not be the same account vpc_name: example #vpc_id: vpc-XXXXXXX # optionally specify VPC ID to use - subnets: # list of public subnet names - - example-dev-a - - example-dev-b security_groups: [] # list of security groups, accepts names or IDs cluster_name: example-cluster family_name: example-task-definition task_definition_revision: "" # integer, but must be presented as a string for Jinja2 task_definition_force_create: false # creates a task definition revision every time if set to true + task_execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable + #task_role_arn: "" # required if you set service_enable_ssm to true task_count: 1 task_minimum_count: 1 task_maximum_count: 4 + # These subnets are usually the subnets created by ce-provision when you made your ECS cluster and must have a NAT gateway for ECR access. + service_subnets: # list of private subnet names + - example-cluster-dev-a + - example-cluster-dev-b # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html service_autoscale_metric_type: ECSServiceAverageCPUUtilization service_autoscale_up_cooldown: 120 @@ -47,7 +50,6 @@ deploy_container: service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API service_force_refresh: false # forces a refresh of all containers if set to true - execution_role_arn: "arn:aws:iam::000000000000:role/ecsTaskExecutionRole" # ARN of the IAM role to run the task as, must have access to the ECR repository if applicable containers: # list of container definitions, see docs: https://docs.ansible.com/ansible/latest/collections/community/aws/ecs_taskdefinition_module.html#parameter-containers - name: example-container essential: true @@ -84,6 +86,9 @@ deploy_container: extra_domains: [] # list of Subject Alternative Name domains and zones ssl_certificate_ARN: "" # optional SSL cert ARN if you imported one into AWS Certificate Manager elb_security_groups: [] # default SG is used if none provided - module supports names or IDs + elb_subnets: # must be public subnets for public facing applications + - example-dev-a + - example-dev-b elb_http_port: 80 elb_https_port: 443 elb_ssl_policy: ELBSecurityPolicy-TLS13-1-2-2021-06 # see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html#describe-ssl-policies diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index de12e5bc..6c68bebd 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -115,11 +115,19 @@ - name: Reset subnets lists. ansible.builtin.set_fact: _aws_ecs_cluster_public_subnets_ids: [] + _aws_ecs_cluster_private_subnets_ids: [] when: deploy_container.aws_ecs.enabled - name: Construct list of public subnet IDs. - ansible.builtin.include_tasks: subnet.yml - with_items: "{{ deploy_container.aws_ecs.subnets }}" + ansible.builtin.include_tasks: subnet-public.yml + with_items: "{{ deploy_container.aws_ecs.elb_subnets }}" + loop_control: + loop_var: subnet + when: deploy_container.aws_ecs.enabled + +- name: Construct list of private subnet IDs. + ansible.builtin.include_tasks: subnet-private.yml + with_items: "{{ deploy_container.aws_ecs.service_subnets }}" loop_control: loop_var: subnet when: deploy_container.aws_ecs.enabled @@ -130,7 +138,8 @@ region: "{{ deploy_container.aws_ecs.region }}" profile: "{{ deploy_container.aws_ecs.aws_profile }}" family: "{{ deploy_container.aws_ecs.family_name }}" - execution_role_arn: "{{ deploy_container.aws_ecs.execution_role_arn }}" + execution_role_arn: "{{ deploy_container.aws_ecs.task_execution_role_arn }}" + task_role_arn: "{{ deploy_container.aws_ecs.task_role_arn | default(omit) }}" containers: "{{ deploy_container.aws_ecs.containers }}" launch_type: "{{ deploy_container.aws_ecs.launch_type }}" cpu: "{{ deploy_container.aws_ecs.cpu | default(omit) }}" @@ -291,7 +300,7 @@ containerPort: "{{ deploy_container.aws_ecs.target_group_port }}" targetGroupArn: "{{ _aws_ecs_target_group.target_group_arn }}" network_configuration: - subnets: "{{ _aws_ecs_cluster_public_subnets_ids }}" + subnets: "{{ _aws_ecs_cluster_private_subnets_ids }}" # internal private subnet security_groups: "{{ deploy_container.aws_ecs.security_groups }}" assign_public_ip: "{{ deploy_container.aws_ecs.service_public_container_ip }}" tags: "{{ deploy_container.aws_ecs.tags }}" diff --git a/roles/deploy_container/tasks/subnet-private.yml b/roles/deploy_container/tasks/subnet-private.yml new file mode 100644 index 00000000..2335cb89 --- /dev/null +++ b/roles/deploy_container/tasks/subnet-private.yml @@ -0,0 +1,13 @@ +- name: Gather private subnet information. + amazon.aws.ec2_vpc_subnet_info: + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + region: "{{ deploy_container.aws_ecs.region }}" + filters: + vpc-id: "{{ _aws_ecs_cluster_vpc_id }}" + tag:Name: "{{ subnet }}" + register: _aws_ecs_cluster_private_subnet + delegate_to: localhost + +- name: Add private subnet to the list. + ansible.builtin.set_fact: + _aws_ecs_cluster_private_subnets_ids: "{{ _aws_ecs_cluster_private_subnets_ids + [ _aws_ecs_cluster_private_subnet.subnets[0].subnet_id ] }}" diff --git a/roles/deploy_container/tasks/subnet.yml b/roles/deploy_container/tasks/subnet-public.yml similarity index 100% rename from roles/deploy_container/tasks/subnet.yml rename to roles/deploy_container/tasks/subnet-public.yml From d86ad0b82d03b1070c54e6e8a4b1197ed5945a98 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 30 Jun 2023 10:52:17 +0200 Subject: [PATCH 044/190] Ecs deployments pr devel (#332) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/deploy_container.md | 3 +++ roles/deploy_container/README.md | 3 +++ 2 files changed, 6 insertions(+) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 8152cedc..f127bb61 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -30,6 +30,9 @@ The full list is: Naturally you can always create custom policies and roles to have tighter access control. This document simply gives you the broad strokes AWS managed policies you can use in conjunction with this Ansible role. +# Peculiarities of AWS ECS +It is worth noting that even if you put your containers on private subnets and configure your apps to use internal addressing, traffic will pass via the public interface. Therefore any safelisting of IP addresses needs to include the IP addresses of the NAT gateways of your private subnets. [More on how this works here.](https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/networking-connecting-vpc.html) + diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 8152cedc..f127bb61 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -30,6 +30,9 @@ The full list is: Naturally you can always create custom policies and roles to have tighter access control. This document simply gives you the broad strokes AWS managed policies you can use in conjunction with this Ansible role. +# Peculiarities of AWS ECS +It is worth noting that even if you put your containers on private subnets and configure your apps to use internal addressing, traffic will pass via the public interface. Therefore any safelisting of IP addresses needs to include the IP addresses of the NAT gateways of your private subnets. [More on how this works here.](https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/networking-connecting-vpc.html) + From 6a2c30d44699041bc942548ee354f18ce99835cd Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 10 Jul 2023 11:13:41 +0200 Subject: [PATCH 045/190] Bug fixes pr devel (#333) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup/database_backup-mysql/tasks/revert-dump.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index 38522af3..4fee5ceb 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -2,7 +2,7 @@ - name: Unpack dump file. ansible.builtin.unarchive: src: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.bz2" - dest: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql" + dest: "/tmp" remote_src: true run_once: true when: previous_build_number > 0 From def3775cb6d0c3f1acb4515493c43b0e10fe2e9a Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 19 Jul 2023 14:44:07 +0200 Subject: [PATCH 046/190] Bug fixes pr devel (#335) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .github/workflows/ce-deploy-lint.yml | 28 ++++++++ ...oy-test.yml => ce-deploy-publish-docs.yml} | 64 ++++--------------- .github/workflows/ce-deploy-test-drupal8.yml | 44 +++++++++++++ .github/workflows/ce-deploy-test-drupal9.yml | 44 +++++++++++++ ce-dev/ansible/common.yml | 2 +- ce-dev/ansible/examples/drupal8/deploy.yml | 33 ++++++++-- ce-dev/ansible/examples/drupal9/deploy.yml | 13 ++-- ce-dev/ansible/examples/localgov/deploy.yml | 49 ++++++++++---- 8 files changed, 201 insertions(+), 76 deletions(-) create mode 100644 .github/workflows/ce-deploy-lint.yml rename .github/workflows/{ce-deploy-test.yml => ce-deploy-publish-docs.yml} (61%) create mode 100644 .github/workflows/ce-deploy-test-drupal8.yml create mode 100644 .github/workflows/ce-deploy-test-drupal9.yml diff --git a/.github/workflows/ce-deploy-lint.yml b/.github/workflows/ce-deploy-lint.yml new file mode 100644 index 00000000..283cd3b9 --- /dev/null +++ b/.github/workflows/ce-deploy-lint.yml @@ -0,0 +1,28 @@ +name: Linting + +# Run this workflow every time a new commit pushed to your repository +on: pull_request + +jobs: + # Set the job key. The key is displayed as the job name + # when a job name is not provided + run-linting: + # Name the Job + name: Linting + # Set the type of machine to run on + runs-on: ubuntu-20.04 + + steps: + # Checks out a copy of your repository on the ubuntu-latest machine + - name: Checkout code + uses: actions/checkout@v2 + + # Linter checks. + - name: Run linters. + if: ${{ always() }} + run: | + sudo apt-get update + sudo apt-get install -y ansible-lint yamllint shellcheck + find ./roles -name "*.yml" | xargs ansible-lint + yamllint ./roles + cd scripts && shellcheck *.sh diff --git a/.github/workflows/ce-deploy-test.yml b/.github/workflows/ce-deploy-publish-docs.yml similarity index 61% rename from .github/workflows/ce-deploy-test.yml rename to .github/workflows/ce-deploy-publish-docs.yml index b93a6d3e..45148600 100644 --- a/.github/workflows/ce-deploy-test.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -1,14 +1,18 @@ -name: Run tests and build docs +name: Publish documentation # Run this workflow every time a new commit pushed to your repository -on: pull_request +on: + pull_request: + types: [closed] + branches: + - 1.x jobs: # Set the job key. The key is displayed as the job name # when a job name is not provided - run-tests: + publish-docs: # Name the Job - name: Run tests against Ansible code base + name: Publish ce-deploy documentation # Set the type of machine to run on runs-on: ubuntu-20.04 @@ -17,15 +21,11 @@ jobs: - name: Checkout code uses: actions/checkout@v2 - # Linter checks. - - name: Run linters. - if: ${{ always() }} + # Configures global Git variables for committing + - name: Configure Git run: | - sudo apt-get update - sudo apt-get install -y ansible-lint yamllint shellcheck - find ./roles -name "*.yml" | xargs ansible-lint - yamllint ./roles - cd scripts && shellcheck *.sh + git config --global user.email "sysadm@codeenigma.com" + git config --global user.name "Code Enigma CI" # Installs the ce-dev stack - name: Install ce-dev @@ -41,46 +41,6 @@ jobs: rm -Rf mkcert curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/1.x/install.sh | /bin/sh -s -- linux - # Configures global Git variables for committing - - name: Configure Git - run: | - git config --global user.email "sysadm@codeenigma.com" - git config --global user.name "Code Enigma CI" - - # Uses the ce-dev stack to run a test Drupal deploy - - name: Run a test ce-dev deploy with Drupal 8 - run: | - ce-dev create -p test -t drupal8 -d ~/test - cd ~/test - ce-dev init - ce-dev start - ce-dev provision - ce-dev deploy - curl https://www.test.local - shell: bash - - - name: Run a test ce-dev deploy with Drupal 9 - run: | - ce-dev create -p testnine -t drupal9 -d ~/testnine - cd ~/testnine - ce-dev init - ce-dev start - ce-dev provision - ce-dev deploy - curl https://www.testnine.local - shell: bash - - - name: Run a test ce-dev deploy with Localgov - run: | - ce-dev create -p testlocalgov -t localgov -d ~/testlocalgov - cd ~/testlocalgov - ce-dev init - ce-dev start - ce-dev provision - ce-dev deploy - curl https://www.testlocalgov.local - shell: bash - # Builds the table of contents for the docs - name: Documentation (build table of contents) if: ${{ github.event.pull_request.base.ref == '1.x' }} diff --git a/.github/workflows/ce-deploy-test-drupal8.yml b/.github/workflows/ce-deploy-test-drupal8.yml new file mode 100644 index 00000000..ec9909a9 --- /dev/null +++ b/.github/workflows/ce-deploy-test-drupal8.yml @@ -0,0 +1,44 @@ +name: Run test for Drupal 8 + +# Run this workflow every time a new commit pushed to your repository +on: pull_request + +jobs: + # Set the job key. The key is displayed as the job name + # when a job name is not provided + run-tests: + # Name the Job + name: Run Drupal 8 test build + # Set the type of machine to run on + runs-on: ubuntu-20.04 + + steps: + # Checks out a copy of your repository on the ubuntu-latest machine + - name: Checkout code + uses: actions/checkout@v2 + + # Installs the ce-dev stack + - name: Install ce-dev + run: | + cd /tmp + wget https://golang.org/dl/go1.16.3.linux-amd64.tar.gz + sudo tar -C /usr/local -xzf go1.16.3.linux-amd64.tar.gz + export PATH=$PATH:/usr/local/go/bin + git clone https://github.com/FiloSottile/mkcert && cd mkcert + go build -ldflags "-X main.Version=$(git describe --tags)" + sudo mv ./mkcert /usr/local/bin && cd ../ + sudo chmod +x /usr/local/bin/mkcert + rm -Rf mkcert + curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/1.x/install.sh | /bin/sh -s -- linux + + # Uses the ce-dev stack to run a test Drupal deploy + - name: Run a test ce-dev deploy with Drupal 8 + run: | + ce-dev create -p test -t drupal8 -d ~/test + cd ~/test + ce-dev init + ce-dev start + ce-dev provision + ce-dev deploy + curl https://www.test.local + shell: bash diff --git a/.github/workflows/ce-deploy-test-drupal9.yml b/.github/workflows/ce-deploy-test-drupal9.yml new file mode 100644 index 00000000..afaaca7a --- /dev/null +++ b/.github/workflows/ce-deploy-test-drupal9.yml @@ -0,0 +1,44 @@ +name: Run test for Drupal 9 + +# Run this workflow every time a new commit pushed to your repository +on: pull_request + +jobs: + # Set the job key. The key is displayed as the job name + # when a job name is not provided + run-tests: + # Name the Job + name: Run Drupal 9 test build + # Set the type of machine to run on + runs-on: ubuntu-20.04 + + steps: + # Checks out a copy of your repository on the ubuntu-latest machine + - name: Checkout code + uses: actions/checkout@v2 + + # Installs the ce-dev stack + - name: Install ce-dev + run: | + cd /tmp + wget https://golang.org/dl/go1.16.3.linux-amd64.tar.gz + sudo tar -C /usr/local -xzf go1.16.3.linux-amd64.tar.gz + export PATH=$PATH:/usr/local/go/bin + git clone https://github.com/FiloSottile/mkcert && cd mkcert + go build -ldflags "-X main.Version=$(git describe --tags)" + sudo mv ./mkcert /usr/local/bin && cd ../ + sudo chmod +x /usr/local/bin/mkcert + rm -Rf mkcert + curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/1.x/install.sh | /bin/sh -s -- linux + + # Uses the ce-dev stack to run a test Drupal deploy + - name: Run a test ce-dev deploy with Drupal 9 + run: | + ce-dev create -p test -t drupal9 -d ~/test + cd ~/test + ce-dev init + ce-dev start + ce-dev provision + ce-dev deploy + curl https://www.test.local + shell: bash diff --git a/ce-dev/ansible/common.yml b/ce-dev/ansible/common.yml index 98babe50..2cffc05e 100644 --- a/ce-dev/ansible/common.yml +++ b/ce-dev/ansible/common.yml @@ -1,5 +1,5 @@ # Common vars for local development. -is_local: yes +is_local: true ce_deploy: username: "ce-dev" own_repository: "https://github.com/codeenigma/ce-deploy.git" diff --git a/ce-dev/ansible/examples/drupal8/deploy.yml b/ce-dev/ansible/examples/drupal8/deploy.yml index 87c6efad..7c8def57 100644 --- a/ce-dev/ansible/examples/drupal8/deploy.yml +++ b/ce-dev/ansible/examples/drupal8/deploy.yml @@ -37,14 +37,35 @@ pre_tasks: # You can safely remove these steps once you have a working composer.json. - name: Download composer file. - get_url: - url: https://raw.githubusercontent.com/drupal/recommended-project/8.8.x/composer.json + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/drupal/recommended-project/8.9.x/composer.json dest: "{{ deploy_path }}/composer.json" - force: no + force: false + - name: Adding composer/installers plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.composer/installers true + working_dir: "{{ deploy_path }}" + - name: Adding drupal/core-composer-scaffold plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.drupal/core-composer-scaffold true + working_dir: "{{ deploy_path }}" + - name: Adding drupal/core-project-message plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.drupal/core-project-message true + working_dir: "{{ deploy_path }}" + - name: Adding dealerdirect/phpcodesniffer-composer-installer plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.dealerdirect/phpcodesniffer-composer-installer true + working_dir: "{{ deploy_path }}" - name: Install drush. - command: - cmd: composer require drush/drush - chdir: "{{ deploy_path }}" + community.general.composer: + command: require + arguments: drush/drush:10.* + working_dir: "{{ deploy_path }}" roles: - _init # Sets some variables the deploy scripts rely on. - composer # Composer install step. diff --git a/ce-dev/ansible/examples/drupal9/deploy.yml b/ce-dev/ansible/examples/drupal9/deploy.yml index deeef9ee..b7579ac4 100644 --- a/ce-dev/ansible/examples/drupal9/deploy.yml +++ b/ce-dev/ansible/examples/drupal9/deploy.yml @@ -37,14 +37,15 @@ pre_tasks: # You can safely remove these steps once you have a working composer.json. - name: Download composer file. - get_url: - url: https://raw.githubusercontent.com/drupal/recommended-project/9.3.x/composer.json + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/drupal/recommended-project/9.5.x/composer.json dest: "{{ deploy_path }}/composer.json" - force: no + force: false - name: Install drush. - command: - cmd: composer require drush/drush:11.* - chdir: "{{ deploy_path }}" + community.general.composer: + command: require + arguments: drush/drush:11.* + working_dir: "{{ deploy_path }}" roles: - _init # Sets some variables the deploy scripts rely on. - composer # Composer install step. diff --git a/ce-dev/ansible/examples/localgov/deploy.yml b/ce-dev/ansible/examples/localgov/deploy.yml index 9844206b..09db8ead 100644 --- a/ce-dev/ansible/examples/localgov/deploy.yml +++ b/ce-dev/ansible/examples/localgov/deploy.yml @@ -37,18 +37,45 @@ pre_tasks: # You can safely remove these steps once you have a working composer.json. - name: Download composer file. - get_url: - url: https://raw.githubusercontent.com/drupal/recommended-project/9.3.x/composer.json - dest: "{{ deploy_path }}/composer.json" - force: false + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/drupal/recommended-project/9.1.x/composer.json + dest: "{{ deploy_path }}/composer.json" + force: false + - name: Adding composer/installers plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.composer/installers true + working_dir: "{{ deploy_path }}" + - name: Adding drupal/core-composer-scaffold plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.drupal/core-composer-scaffold true + working_dir: "{{ deploy_path }}" + - name: Adding drupal/core-project-message plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.drupal/core-project-message true + working_dir: "{{ deploy_path }}" + - name: Adding dealerdirect/phpcodesniffer-composer-installer plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.dealerdirect/phpcodesniffer-composer-installer true + working_dir: "{{ deploy_path }}" + - name: Adding cweagans/composer-patches plugin to composer config. + community.general.composer: + command: config + arguments: allow-plugins.cweagans/composer-patches true + working_dir: "{{ deploy_path }}" - name: Install drush. - command: - cmd: composer require drush/drush:11.* - chdir: "{{ deploy_path }}" + community.general.composer: + command: require + arguments: drush/drush:10.* + working_dir: "{{ deploy_path }}" - name: Install localgov. - command: - cmd: composer require localgovdrupal/localgov - chdir: "{{ deploy_path }}" + community.general.composer: + command: require + arguments: localgovdrupal/localgov + working_dir: "{{ deploy_path }}" roles: - _init # Sets some variables the deploy scripts rely on. - composer # Composer install step. @@ -56,4 +83,4 @@ - config_generate # Generates settings.php # - sync/database_sync # Grab database from a remote server. - database_apply # Run drush updb and config import. - - _exit # Some common housekeeping. \ No newline at end of file + - _exit # Some common housekeeping. From 00796d4914096138e0b98ce94031010ac72adcd6 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 20 Jul 2023 20:15:17 +0300 Subject: [PATCH 047/190] Default to gzip pr devel (#337) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- .../templates/regular-backups.sh.j2 | 8 ++++---- .../database_backup-mysql/tasks/cleanup-dump.yml | 2 +- .../database_backup-mysql/tasks/deploy-dump.yml | 2 +- .../database_backup-mysql/tasks/revert-dump.yml | 2 +- .../database_sync/database_sync-mysql/defaults/main.yml | 2 +- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/roles/cron/cron_database_backup/cron_database_backup-mysql/templates/regular-backups.sh.j2 b/roles/cron/cron_database_backup/cron_database_backup-mysql/templates/regular-backups.sh.j2 index 33c27a5a..691163f5 100644 --- a/roles/cron/cron_database_backup/cron_database_backup-mysql/templates/regular-backups.sh.j2 +++ b/roles/cron/cron_database_backup/cron_database_backup-mysql/templates/regular-backups.sh.j2 @@ -6,20 +6,20 @@ DBHOST='{{ database.host }}' DB_NAME='{{ database.original.database }}' CURRENT_DBNAME='{{ database.name }}' TARGET_DIR="{{ cron_mysql_backup.dumps_directory }}/$DBHOST" -TARBALL="$DB_NAME-$(date -Iseconds).sql.bz2" +TARBALL="$DB_NAME-$(date -Iseconds).sql.gz" KEEP=$(({{ database.original.backup.keep | default(cron_mysql_backup.keep) }}+1)) backup(){ mysqldump {{ cron_mysql_backup.mysqldump_params }} \ - -u"$DBUSER" -p"$DBPASSWORD" -h"$DBHOST" "$CURRENT_DBNAME" | bzip2 > "$TARGET_DIR/$TARBALL" + -u"$DBUSER" -p"$DBPASSWORD" -h"$DBHOST" "$CURRENT_DBNAME" | gzip > "$TARGET_DIR/$TARBALL" ln -sfn "$TARGET_DIR/$TARBALL" "$TARGET_DIR/$DB_NAME" } cleanup(){ - if [ "$(find "$TARGET_DIR" -name '*sql.bz2' | wc -l)" -lt "$KEEP" ]; then + if [ "$(find "$TARGET_DIR" -name '*.sql.*' | wc -l)" -lt "$KEEP" ]; then return 0 fi - rm "$(find $TARGET_DIR -name '*.sql.bz2' | sort | head -n 1)" + rm "$(find $TARGET_DIR -name '*.sql.*' | sort | head -n 1)" cleanup } diff --git a/roles/database_backup/database_backup-mysql/tasks/cleanup-dump.yml b/roles/database_backup/database_backup-mysql/tasks/cleanup-dump.yml index 2609cabe..734aad4d 100644 --- a/roles/database_backup/database_backup-mysql/tasks/cleanup-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/cleanup-dump.yml @@ -2,7 +2,7 @@ # We assume it's safe to only go back 50 build back. - name: Delete mysql dumps. ansible.builtin.file: - path: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ item }}.sql.bz2" + path: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ item }}.sql.*" state: absent with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - mysql_backup.keep, 0] | max }} run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml index 9f6f6959..9c11857f 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml @@ -23,7 +23,7 @@ run_once: true - name: Take a database dump. - ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.credentials_file }} {{ mysql_backup.mysqldump_params }} {{ database.database }} | bzip2 > {{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.bz2" + ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.credentials_file }} {{ mysql_backup.mysqldump_params }} {{ database.database }} | gzip > {{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz" args: executable: /bin/bash when: previous_build_number > 0 diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index 4fee5ceb..08059e30 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -1,7 +1,7 @@ --- - name: Unpack dump file. ansible.builtin.unarchive: - src: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.bz2" + src: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz" dest: "/tmp" remote_src: true run_once: true diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index 88143a7b..98ac19bf 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -2,7 +2,7 @@ mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here. cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes. - archival_method: "bzip2" # oprions are "bzip2" or "gzip". + archival_method: "gzip" # oprions are "bzip2" or "gzip". databases: - source: # Name of the database to take a dump from. From c69d11610eecb4d73e20d08630339950d47a3d46 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 4 Aug 2023 17:04:25 +0300 Subject: [PATCH 048/190] Update for database names handling pr devel (#339) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- roles/sync/database_sync/database_sync-mysql/tasks/sync.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 7e52bd07..25b5bfb8 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -157,14 +157,14 @@ - name: Drop target database. ansible.builtin.command: - cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'drop database if exists {{ mysql_sync_target_database }};'" + cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'drop database if exists `{{ mysql_sync_target_database }}`;'" - name: Recreate target database. ansible.builtin.command: - cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'create database {{ mysql_sync_target_database }};'" + cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'create database `{{ mysql_sync_target_database }}`;'" - name: Repopulate database from dump. - ansible.builtin.shell: "mysql --defaults-extra-file={{ database.target.credentials_file }} {{ mysql_sync_target_database }} < {{ mysql_sync_target_dump_unpacked_path }}" + ansible.builtin.shell: "mysql --defaults-extra-file={{ database.target.credentials_file }} `{{ mysql_sync_target_database }}` < {{ mysql_sync_target_dump_unpacked_path }}" args: executable: /bin/bash From 695a1cf6fe5aa3b487267b25291a79af1b2d76bf Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 4 Aug 2023 19:44:51 +0300 Subject: [PATCH 049/190] Update for database names handling fix pr devel (#341) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey --- roles/sync/database_sync/database_sync-mysql/tasks/sync.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 25b5bfb8..3f94e47d 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -164,7 +164,7 @@ cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'create database `{{ mysql_sync_target_database }}`;'" - name: Repopulate database from dump. - ansible.builtin.shell: "mysql --defaults-extra-file={{ database.target.credentials_file }} `{{ mysql_sync_target_database }}` < {{ mysql_sync_target_dump_unpacked_path }}" + ansible.builtin.shell: "mysql --defaults-extra-file={{ database.target.credentials_file }} {{ mysql_sync_target_database }} < {{ mysql_sync_target_dump_unpacked_path }}" args: executable: /bin/bash From a1d8d2eff8d149eb5f6f726d92043d528a7be646 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 10 Aug 2023 17:10:17 +0200 Subject: [PATCH 050/190] Feature branching pr devel (#345) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/_init.md | 8 ++++++ .../sync/database_sync/database_sync-mysql.md | 2 +- roles/_init/README.md | 8 ++++++ roles/_init/defaults/main.yml | 8 ++++++ .../database_apply-drupal8/tasks/main.yml | 26 +++++++++++++++++++ .../database_sync-mysql/README.md | 2 +- 6 files changed, 52 insertions(+), 2 deletions(-) diff --git a/docs/roles/_init.md b/docs/roles/_init.md index edf1f5d3..8118c9ea 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -34,9 +34,17 @@ drupal: cron: - minute: "*/{{ 10 | random(start=1) }}" job: cron + feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment + # For syncing database and files on a feature branch initial build + mysql_sync: # see sync/database_sync for docs + databases: [] + files_sync: # see sync/files_sync for docs + cleanup: true + directories: [] mautic: image_path: "media/images" force_install: false +# Used for custom build time tools like cachetool bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 diff --git a/docs/roles/sync/database_sync/database_sync-mysql.md b/docs/roles/sync/database_sync/database_sync-mysql.md index d70d770e..c0c2954b 100644 --- a/docs/roles/sync/database_sync/database_sync-mysql.md +++ b/docs/roles/sync/database_sync/database_sync-mysql.md @@ -7,7 +7,7 @@ Sync MySQL databases between environments. mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here. cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes. - archival_method: "bzip2" # oprions are "bzip2" or "gzip". + archival_method: "gzip" # oprions are "bzip2" or "gzip". databases: - source: # Name of the database to take a dump from. diff --git a/roles/_init/README.md b/roles/_init/README.md index edf1f5d3..8118c9ea 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -34,9 +34,17 @@ drupal: cron: - minute: "*/{{ 10 | random(start=1) }}" job: cron + feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment + # For syncing database and files on a feature branch initial build + mysql_sync: # see sync/database_sync for docs + databases: [] + files_sync: # see sync/files_sync for docs + cleanup: true + directories: [] mautic: image_path: "media/images" force_install: false +# Used for custom build time tools like cachetool bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 diff --git a/roles/_init/defaults/main.yml b/roles/_init/defaults/main.yml index 57ee7e44..2fad2e47 100644 --- a/roles/_init/defaults/main.yml +++ b/roles/_init/defaults/main.yml @@ -25,9 +25,17 @@ drupal: cron: - minute: "*/{{ 10 | random(start=1) }}" job: cron + feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment + # For syncing database and files on a feature branch initial build + mysql_sync: # see sync/database_sync for docs + databases: [] + files_sync: # see sync/files_sync for docs + cleanup: true + directories: [] mautic: image_path: "media/images" force_install: false +# Used for custom build time tools like cachetool bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 diff --git a/roles/database_apply/database_apply-drupal8/tasks/main.yml b/roles/database_apply/database_apply-drupal8/tasks/main.yml index c423fad9..7faf684f 100644 --- a/roles/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/database_apply/database_apply-drupal8/tasks/main.yml @@ -25,6 +25,32 @@ when: (previous_build_number == 0) or (site.force_install is defined and site.force_install) register: _drush_output +- name: Sync database. + ansible.builtin.import_role: + name: sync/database_sync + vars: + mysql_sync: "{{ site.mysql_sync }}" + with_items: "{{ drupal.sites }}" + loop_control: + loop_var: site + when: + - (previous_build_number == 0) or (site.force_install is defined and site.force_install) + - site.feature_branch | default(false) + - site.mysql_sync | length > 0 + +- name: Sync files. + ansible.builtin.import_role: + name: sync/files_sync + vars: + files_sync: "{{ site.files_sync }}" + with_items: "{{ drupal.sites }}" + loop_control: + loop_var: site + when: + - (previous_build_number == 0) or (site.force_install is defined and site.force_install) + - site.feature_branch | default(false) + - site.files_sync | length > 0 + - name: Show drush output. ansible.builtin.debug: msg: "{{ _drush_output }}" diff --git a/roles/sync/database_sync/database_sync-mysql/README.md b/roles/sync/database_sync/database_sync-mysql/README.md index d70d770e..c0c2954b 100644 --- a/roles/sync/database_sync/database_sync-mysql/README.md +++ b/roles/sync/database_sync/database_sync-mysql/README.md @@ -7,7 +7,7 @@ Sync MySQL databases between environments. mysql_sync: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here. cleanup: true # if false leaves tmp database dump on deploy server for debugging purposes. - archival_method: "bzip2" # oprions are "bzip2" or "gzip". + archival_method: "gzip" # oprions are "bzip2" or "gzip". databases: - source: # Name of the database to take a dump from. From fdb0f28329aeba39cdf28597db6057464abb63d0 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 10 Aug 2023 18:25:30 +0200 Subject: [PATCH 051/190] Feature branching pr devel (#347) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/database_backup/database_backup-mysql.md | 4 ++-- roles/database_apply/database_apply-drupal8/tasks/main.yml | 4 ++-- roles/database_backup/database_backup-mysql/README.md | 4 ++-- roles/database_backup/database_backup-mysql/defaults/main.yml | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/roles/database_backup/database_backup-mysql.md b/docs/roles/database_backup/database_backup-mysql.md index b52413bf..c52620f6 100644 --- a/docs/roles/database_backup/database_backup-mysql.md +++ b/docs/roles/database_backup/database_backup-mysql.md @@ -22,8 +22,8 @@ mysql_backup: # This is useful for locked-down setups where you do not have GRANT permissions. credentials_handling: rotate databases: - - database: "{{ project_name }}_{{ build_type }}" - user: "{{ project_name }}_{{ build_type }}" + - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names + user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" ``` diff --git a/roles/database_apply/database_apply-drupal8/tasks/main.yml b/roles/database_apply/database_apply-drupal8/tasks/main.yml index 7faf684f..7faa03c7 100644 --- a/roles/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/database_apply/database_apply-drupal8/tasks/main.yml @@ -26,7 +26,7 @@ register: _drush_output - name: Sync database. - ansible.builtin.import_role: + ansible.builtin.include_role: name: sync/database_sync vars: mysql_sync: "{{ site.mysql_sync }}" @@ -39,7 +39,7 @@ - site.mysql_sync | length > 0 - name: Sync files. - ansible.builtin.import_role: + ansible.builtin.include_role: name: sync/files_sync vars: files_sync: "{{ site.files_sync }}" diff --git a/roles/database_backup/database_backup-mysql/README.md b/roles/database_backup/database_backup-mysql/README.md index b52413bf..c52620f6 100644 --- a/roles/database_backup/database_backup-mysql/README.md +++ b/roles/database_backup/database_backup-mysql/README.md @@ -22,8 +22,8 @@ mysql_backup: # This is useful for locked-down setups where you do not have GRANT permissions. credentials_handling: rotate databases: - - database: "{{ project_name }}_{{ build_type }}" - user: "{{ project_name }}_{{ build_type }}" + - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names + user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" ``` diff --git a/roles/database_backup/database_backup-mysql/defaults/main.yml b/roles/database_backup/database_backup-mysql/defaults/main.yml index f7e070e9..1cbc2e11 100644 --- a/roles/database_backup/database_backup-mysql/defaults/main.yml +++ b/roles/database_backup/database_backup-mysql/defaults/main.yml @@ -17,6 +17,6 @@ mysql_backup: # This is useful for locked-down setups where you do not have GRANT permissions. credentials_handling: rotate databases: - - database: "{{ project_name }}_{{ build_type }}" - user: "{{ project_name }}_{{ build_type }}" + - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names + user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" From cafd3b545c47a505965317bc3cc156a11a8e37aa Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 11 Aug 2023 13:16:35 +0200 Subject: [PATCH 052/190] Feature branching pr devel (#349) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. * Adding syncing for feature branches. (#346) * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- docs/roles/_init.md | 16 ++++++++++------ docs/roles/composer.md | 3 ++- docs/roles/config_generate.md | 2 +- .../config_generate/config_generate-drupal8.md | 8 ++++++++ docs/roles/npm.md | 3 ++- roles/_init/README.md | 16 ++++++++++------ roles/_init/defaults/main.yml | 16 ++++++++++------ roles/composer/README.md | 3 ++- roles/composer/defaults/main.yml | 4 ++-- roles/config_generate/README.md | 2 +- .../config_generate-drupal8/README.md | 8 ++++++++ .../config_generate-drupal8/defaults/main.yml | 4 ++++ .../config_generate-drupal8/tasks/drush.yml | 5 +++-- .../config_generate-drupal8/tasks/settings.yml | 5 +++-- roles/config_generate/defaults/main.yml | 2 +- roles/live_symlink/defaults/main.yml | 4 ++-- roles/npm/README.md | 3 ++- roles/npm/defaults/main.yml | 4 ++-- 18 files changed, 73 insertions(+), 35 deletions(-) create mode 100644 roles/config_generate/config_generate-drupal8/defaults/main.yml diff --git a/docs/roles/_init.md b/docs/roles/_init.md index 8118c9ea..d561aac2 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -35,12 +35,16 @@ drupal: - minute: "*/{{ 10 | random(start=1) }}" job: cron feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment - # For syncing database and files on a feature branch initial build - mysql_sync: # see sync/database_sync for docs - databases: [] - files_sync: # see sync/files_sync for docs - cleanup: true - directories: [] + # For syncing database and files on a feature branch initial build - include all variables if used + mysql_sync: {} # see sync/database_sync for docs + # mysqldump_params: "{{ _mysqldump_params }}" + # cleanup: true + # archival_method: gzip + # databases: [] + files_sync: {} # see sync/files_sync for docs + # unique_workspace: false + # cleanup: true + # directories: [] mautic: image_path: "media/images" force_install: false diff --git a/docs/roles/composer.md b/docs/roles/composer.md index 17144004..74765170 100644 --- a/docs/roles/composer.md +++ b/docs/roles/composer.md @@ -14,7 +14,7 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' @@ -24,6 +24,7 @@ composer: # src: name of a template, in the "templates" dir relative to your playbook. # dest: can only be relative to the root of your repository (eg. 'www/config.php', 'var/mysettings.php') templates: [] + ``` diff --git a/docs/roles/config_generate.md b/docs/roles/config_generate.md index 5428dd8b..fe7aecfe 100644 --- a/docs/roles/config_generate.md +++ b/docs/roles/config_generate.md @@ -11,7 +11,7 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/docs/roles/config_generate/config_generate-drupal8.md b/docs/roles/config_generate/config_generate-drupal8.md index 20e32b5b..75382886 100644 --- a/docs/roles/config_generate/config_generate-drupal8.md +++ b/docs/roles/config_generate/config_generate-drupal8.md @@ -4,4 +4,12 @@ Generates settings.php file for Drupal 8. +## Default variables +```yaml +config_generate_drupal: + # If you need to specify an exact file name for a template file to use for a build, you can do it here. + drupal_settings_file_name: settings.php + drush_settings_file_name: drush.yml +``` + diff --git a/docs/roles/npm.md b/docs/roles/npm.md index 385e1c84..97f06cdc 100644 --- a/docs/roles/npm.md +++ b/docs/roles/npm.md @@ -18,7 +18,7 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' @@ -28,6 +28,7 @@ npm: # src: name of a template, in the "templates" dir relative to your playbook. # dest: can only be relative to the root of your repository (eg. 'www/config.php', 'var/mysettings.php') templates: [] + ``` diff --git a/roles/_init/README.md b/roles/_init/README.md index 8118c9ea..d561aac2 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -35,12 +35,16 @@ drupal: - minute: "*/{{ 10 | random(start=1) }}" job: cron feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment - # For syncing database and files on a feature branch initial build - mysql_sync: # see sync/database_sync for docs - databases: [] - files_sync: # see sync/files_sync for docs - cleanup: true - directories: [] + # For syncing database and files on a feature branch initial build - include all variables if used + mysql_sync: {} # see sync/database_sync for docs + # mysqldump_params: "{{ _mysqldump_params }}" + # cleanup: true + # archival_method: gzip + # databases: [] + files_sync: {} # see sync/files_sync for docs + # unique_workspace: false + # cleanup: true + # directories: [] mautic: image_path: "media/images" force_install: false diff --git a/roles/_init/defaults/main.yml b/roles/_init/defaults/main.yml index 2fad2e47..5fcee089 100644 --- a/roles/_init/defaults/main.yml +++ b/roles/_init/defaults/main.yml @@ -26,12 +26,16 @@ drupal: - minute: "*/{{ 10 | random(start=1) }}" job: cron feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment - # For syncing database and files on a feature branch initial build - mysql_sync: # see sync/database_sync for docs - databases: [] - files_sync: # see sync/files_sync for docs - cleanup: true - directories: [] + # For syncing database and files on a feature branch initial build - include all variables if used + mysql_sync: {} # see sync/database_sync for docs + # mysqldump_params: "{{ _mysqldump_params }}" + # cleanup: true + # archival_method: gzip + # databases: [] + files_sync: {} # see sync/files_sync for docs + # unique_workspace: false + # cleanup: true + # directories: [] mautic: image_path: "media/images" force_install: false diff --git a/roles/composer/README.md b/roles/composer/README.md index 17144004..74765170 100644 --- a/roles/composer/README.md +++ b/roles/composer/README.md @@ -14,7 +14,7 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' @@ -24,6 +24,7 @@ composer: # src: name of a template, in the "templates" dir relative to your playbook. # dest: can only be relative to the root of your repository (eg. 'www/config.php', 'var/mysettings.php') templates: [] + ``` diff --git a/roles/composer/defaults/main.yml b/roles/composer/defaults/main.yml index 2e6eec4a..d0dfbd1d 100644 --- a/roles/composer/defaults/main.yml +++ b/roles/composer/defaults/main.yml @@ -7,7 +7,7 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' @@ -16,4 +16,4 @@ composer: # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. # dest: can only be relative to the root of your repository (eg. 'www/config.php', 'var/mysettings.php') - templates: [] \ No newline at end of file + templates: [] diff --git a/roles/config_generate/README.md b/roles/config_generate/README.md index 5428dd8b..fe7aecfe 100644 --- a/roles/config_generate/README.md +++ b/roles/config_generate/README.md @@ -11,7 +11,7 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/config_generate/config_generate-drupal8/README.md b/roles/config_generate/config_generate-drupal8/README.md index 20e32b5b..75382886 100644 --- a/roles/config_generate/config_generate-drupal8/README.md +++ b/roles/config_generate/config_generate-drupal8/README.md @@ -4,4 +4,12 @@ Generates settings.php file for Drupal 8. +## Default variables +```yaml +config_generate_drupal: + # If you need to specify an exact file name for a template file to use for a build, you can do it here. + drupal_settings_file_name: settings.php + drush_settings_file_name: drush.yml +``` + diff --git a/roles/config_generate/config_generate-drupal8/defaults/main.yml b/roles/config_generate/config_generate-drupal8/defaults/main.yml new file mode 100644 index 00000000..a6b7634d --- /dev/null +++ b/roles/config_generate/config_generate-drupal8/defaults/main.yml @@ -0,0 +1,4 @@ +config_generate_drupal: + # If you need to specify an exact file name for a template file to use for a build, you can do it here. + drupal_settings_file_name: settings.php + drush_settings_file_name: drush.yml \ No newline at end of file diff --git a/roles/config_generate/config_generate-drupal8/tasks/drush.yml b/roles/config_generate/config_generate-drupal8/tasks/drush.yml index f4b7ae7a..0bedd46d 100644 --- a/roles/config_generate/config_generate-drupal8/tasks/drush.yml +++ b/roles/config_generate/config_generate-drupal8/tasks/drush.yml @@ -4,6 +4,7 @@ src: '{{ item }}' dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/drush.yml" with_first_found: - - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.drush.yml.j2" - - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.drush.yml" + - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.{{ config_generate_drupal.drush_settings_file_name }}.j2" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.{{ config_generate_drupal.drush_settings_file_name }}" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ config_generate_drupal.drush_settings_file_name }}" - "drush.yml.j2" \ No newline at end of file diff --git a/roles/config_generate/config_generate-drupal8/tasks/settings.yml b/roles/config_generate/config_generate-drupal8/tasks/settings.yml index ebf182e8..6f698ba5 100644 --- a/roles/config_generate/config_generate-drupal8/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal8/tasks/settings.yml @@ -23,6 +23,7 @@ src: "{{ item }}" dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/settings.php" with_first_found: - - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php.j2" - - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.settings.php" + - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.{{ config_generate_drupal.drupal_settings_file_name }}.j2" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.{{ config_generate_drupal.drupal_settings_file_name }}" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ config_generate_drupal.drupal_settings_file_name }}" - "settings.php.j2" diff --git a/roles/config_generate/defaults/main.yml b/roles/config_generate/defaults/main.yml index 5258730e..8f1987eb 100644 --- a/roles/config_generate/defaults/main.yml +++ b/roles/config_generate/defaults/main.yml @@ -3,7 +3,7 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/live_symlink/defaults/main.yml b/roles/live_symlink/defaults/main.yml index 4b9fb449..eca23ad6 100644 --- a/roles/live_symlink/defaults/main.yml +++ b/roles/live_symlink/defaults/main.yml @@ -6,7 +6,7 @@ live_symlink: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' @@ -15,4 +15,4 @@ live_symlink: # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. # dest: can only be relative to the root of your repository (eg. 'www/config.php', 'var/mysettings.php') - templates: [] \ No newline at end of file + templates: [] diff --git a/roles/npm/README.md b/roles/npm/README.md index 385e1c84..97f06cdc 100644 --- a/roles/npm/README.md +++ b/roles/npm/README.md @@ -18,7 +18,7 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' @@ -28,6 +28,7 @@ npm: # src: name of a template, in the "templates" dir relative to your playbook. # dest: can only be relative to the root of your repository (eg. 'www/config.php', 'var/mysettings.php') templates: [] + ``` diff --git a/roles/npm/defaults/main.yml b/roles/npm/defaults/main.yml index be6bdc3f..a29fcac7 100644 --- a/roles/npm/defaults/main.yml +++ b/roles/npm/defaults/main.yml @@ -11,7 +11,7 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: wether to create the target if it does not exists. + # create: whether to create the target if it does not exists. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' @@ -20,4 +20,4 @@ npm: # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. # dest: can only be relative to the root of your repository (eg. 'www/config.php', 'var/mysettings.php') - templates: [] \ No newline at end of file + templates: [] From 22ea4d8d67347a2dbc000adecb5cc65633b2d608 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matej=20=C5=A0tajduhar?= <30931414+matej5@users.noreply.github.com> Date: Wed, 13 Sep 2023 12:05:32 +0200 Subject: [PATCH 053/190] R66858 adding variable to skip long running tasks pr devel (#351) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * r66858-adding-variable-to-skip-long-running-tasks --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/defaults/main.yml | 2 ++ roles/deploy_code/tasks/cleanup.yml | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/roles/deploy_code/defaults/main.yml b/roles/deploy_code/defaults/main.yml index f9b84739..7b68c3a7 100644 --- a/roles/deploy_code/defaults/main.yml +++ b/roles/deploy_code/defaults/main.yml @@ -35,6 +35,8 @@ deploy_code: service_action: reload # Trigger an API call to rebuild infra after a deploy, e.g. if you need to repack an AMI. rebuild_infra: false + # Used to skip tasks to fix ownership and permissions, drupal needs this set to true by default + fix_cleanup_perms: true # Details of API call to trigger. See api_call role. api_call: type: gitlab diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index b3a4f132..870f86b7 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -4,7 +4,9 @@ cmd: "if [ -d {{ deploy_path_prefix }}{{ item }} ]; then chmod -R 777 {{ deploy_path_prefix }}{{ item }}; fi" with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - deploy_code.keep, 0] | max }} become: true - when: "www_user != deploy_user" + when: + - "www_user != deploy_user" + - deploy_code.fix_cleanup_perms - name: Ensure permissions are set on deploy directory. ansible.builtin.shell: From fcccd626cef5a321b05d72456626c03273e68324 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 13 Sep 2023 19:46:51 +0200 Subject: [PATCH 054/190] Opcache docs (#353) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Fixing documentation for opcache role. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/cache_clear/cache_clear-opcache/defaults/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/cache_clear/cache_clear-opcache/defaults/main.yml b/roles/cache_clear/cache_clear-opcache/defaults/main.yml index 222325ce..94e1cb22 100644 --- a/roles/cache_clear/cache_clear-opcache/defaults/main.yml +++ b/roles/cache_clear/cache_clear-opcache/defaults/main.yml @@ -4,7 +4,7 @@ cache_clear_opcache: # eg. # --fcgi=127.0.0.1:9000 # Leave blank to use /etc/cachetool.yml - # adapter: "127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. + # adapter: "--fcgi=127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. # Bins to clear. clear_opcache: true clear_apcu: false From a3d2881f4a5756851c36c2b6e31c4b3db88dda7c Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 13:22:52 +0200 Subject: [PATCH 055/190] Improved database handling pr devel (#357) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup/database_backup-mysql.md | 1 + .../database_backup-mysql/README.md | 1 + .../database_backup-mysql/defaults/main.yml | 1 + .../tasks/deploy-rolling.yml | 45 ++++++++++++------- .../database_backup-mysql/tasks/deploy.yml | 10 ++++- 5 files changed, 42 insertions(+), 16 deletions(-) diff --git a/docs/roles/database_backup/database_backup-mysql.md b/docs/roles/database_backup/database_backup-mysql.md index c52620f6..3a7ec471 100644 --- a/docs/roles/database_backup/database_backup-mysql.md +++ b/docs/roles/database_backup/database_backup-mysql.md @@ -25,6 +25,7 @@ mysql_backup: - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" + #handling: static # optional override to the main handling method on a per database basis ``` diff --git a/roles/database_backup/database_backup-mysql/README.md b/roles/database_backup/database_backup-mysql/README.md index c52620f6..3a7ec471 100644 --- a/roles/database_backup/database_backup-mysql/README.md +++ b/roles/database_backup/database_backup-mysql/README.md @@ -25,6 +25,7 @@ mysql_backup: - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" + #handling: static # optional override to the main handling method on a per database basis ``` diff --git a/roles/database_backup/database_backup-mysql/defaults/main.yml b/roles/database_backup/database_backup-mysql/defaults/main.yml index 1cbc2e11..777cc6f9 100644 --- a/roles/database_backup/database_backup-mysql/defaults/main.yml +++ b/roles/database_backup/database_backup-mysql/defaults/main.yml @@ -20,3 +20,4 @@ mysql_backup: - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" + #handling: static # optional override to the main handling method on a per database basis diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index aec45596..6a264be1 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -4,26 +4,41 @@ _mysql_build_database_name: "{{ database.database }}_{{ build_number }}" - ansible.builtin.set_fact: _mysql_previous_build_database_name: "{{ database.database }}_{{ previous_build_number }}" - # Note: we don't use the mysql_db Ansible module on purpose. - # If database already exists, we want to fail and not override it - # with previous build. - # @TODO fix this so we check if the database exists and exit with - # the proper plugin instead of using command. -- name: Create new database. - ansible.builtin.command: mysql --defaults-extra-file={{ database.credentials_file }} -e "CREATE DATABASE `{{ _mysql_build_database_name }}`;" + +- name: Check if the new database name exists already. + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_build_database_name }}" + register: _build_db_status + failed_when: _build_db_status.rc == 0 # we want the build to fail if the database exists + run_once: true + +- name: Create a new database. + community.mysql.mysql_db: + name: "{{ _mysql_build_database_name }}" + state: present + config_file: "{{ database.credentials_file }}" + config_overrides_defaults: true run_once: true -#- name: Create a new database. -# community.mysql.mysql_db: -# name: "{{ _mysql_build_database_name }}" -# state: present -# config_file: "{{ database.credentials_file }}" -# config_overrides_defaults: true -# run_once: true +- name: Check if the previous database exists. + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_previous_build_database_name }}" + register: _previous_db_status + failed_when: _previous_db_status.rc > 1 # only fail if we get an unexpected exitcode + run_once: true +# Importing with shell for speed, we cannot import with mysql_db unless we create a dump file first - name: Populate new database. ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.credentials_file }} {{ mysql_backup.mysqldump_params }} {{ _mysql_previous_build_database_name }} | mysql --defaults-extra-file={{ database.credentials_file }} {{ _mysql_build_database_name }}" args: executable: /bin/bash - when: previous_build_number > 0 + when: + - previous_build_number > 0 + - _previous_db_status.rc == 0 # only run if we have a database + run_once: true + +# Making it clear if we skipped the population of the new database +- name: Populating database skipped. + ansible.builtin.debug: "### Attention - the new database was NOT populated!" + when: + - previous_build_number > 0 + - _previous_db_status.rc != 0 run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy.yml b/roles/database_backup/database_backup-mysql/tasks/deploy.yml index 4160806c..6e60a96d 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy.yml @@ -82,7 +82,15 @@ _mysql_build_password: "{{ lookup('password', '/dev/shm/{{ project_name }}_{{ build_type }}_{{ build_number }}') }}" when: mysql_backup.credentials_handling == 'rotate' -- ansible.builtin.include_tasks: "deploy-{{ mysql_backup.handling }}.yml" +- ansible.builtin.set_fact: + _mysql_handling: "{{ mysql_backup.handling }}" +# If we have a specific instruction for handling this database differently, use it. +- ansible.builtin.set_fact: + _mysql_handling: "{{ mysql_backup.database.handling }}" + when: + - mysql_backup.database.handling is defined + - mysql_backup.database.handling | length > 0 +- ansible.builtin.include_tasks: "deploy-{{ _mysql_handling }}.yml" # We append privileges instead of replacing, # to allow this role to be looped over, From 917f91c4f3a6231c68ec68735148acf0d0375c5b Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 14:06:02 +0200 Subject: [PATCH 056/190] Improved database handling pr devel (#359) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup-mysql/tasks/deploy-rolling.yml | 3 ++- .../database_backup-mysql/tasks/deploy.yml | 9 ++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index 6a264be1..d5d9f941 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -37,7 +37,8 @@ # Making it clear if we skipped the population of the new database - name: Populating database skipped. - ansible.builtin.debug: "### Attention - the new database was NOT populated!" + ansible.builtin.debug: + msg: "### Attention - the new database was NOT populated!" when: - previous_build_number > 0 - _previous_db_status.rc != 0 diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy.yml b/roles/database_backup/database_backup-mysql/tasks/deploy.yml index 6e60a96d..f6d75658 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy.yml @@ -82,15 +82,18 @@ _mysql_build_password: "{{ lookup('password', '/dev/shm/{{ project_name }}_{{ build_type }}_{{ build_number }}') }}" when: mysql_backup.credentials_handling == 'rotate' -- ansible.builtin.set_fact: +- name: Set database handling type. + ansible.builtin.set_fact: _mysql_handling: "{{ mysql_backup.handling }}" # If we have a specific instruction for handling this database differently, use it. -- ansible.builtin.set_fact: +- name: Override database handling type for this database, if specified. + ansible.builtin.set_fact: _mysql_handling: "{{ mysql_backup.database.handling }}" when: - mysql_backup.database.handling is defined - mysql_backup.database.handling | length > 0 -- ansible.builtin.include_tasks: "deploy-{{ _mysql_handling }}.yml" +- name: Execute backup tasks. + ansible.builtin.include_tasks: "deploy-{{ _mysql_handling }}.yml" # We append privileges instead of replacing, # to allow this role to be looped over, From f20fb2b4f42d8e5bfb03f5cf5599c5d2beeddd9e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 14:46:55 +0200 Subject: [PATCH 057/190] Improved database handling pr devel (#360) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup-mysql/tasks/deploy-rolling.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index d5d9f941..5a82a644 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -10,6 +10,8 @@ register: _build_db_status failed_when: _build_db_status.rc == 0 # we want the build to fail if the database exists run_once: true + args: + executable: /bin/bash - name: Create a new database. community.mysql.mysql_db: @@ -24,6 +26,8 @@ register: _previous_db_status failed_when: _previous_db_status.rc > 1 # only fail if we get an unexpected exitcode run_once: true + args: + executable: /bin/bash # Importing with shell for speed, we cannot import with mysql_db unless we create a dump file first - name: Populate new database. From 483ba4575288e112bb92c49c610fe0e9385affc9 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 15:07:42 +0200 Subject: [PATCH 058/190] Improved database handling pr devel (#361) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup-mysql/tasks/deploy-dump.yml | 13 ++++++++++++- .../database_backup-mysql/tasks/deploy-none.yml | 11 ++++++++++- .../database_backup-mysql/tasks/deploy-rolling.yml | 2 +- .../database_backup-mysql/tasks/deploy.yml | 2 +- 4 files changed, 24 insertions(+), 4 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml index 9c11857f..3bd7e69e 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml @@ -4,6 +4,15 @@ - ansible.builtin.set_fact: _mysql_previous_build_database_name: "{{ database.database }}" +- name: Check if the database exists already. + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_build_database_name }}" + register: _build_db_status + failed_when: _build_db_status.rc > 1 # only exit on abnormal errors + run_once: true + args: + executable: /bin/bash + +# Create database if this is an initial build or it doesn't exist yet - name: Create initial database. community.mysql.mysql_db: name: "{{ _mysql_build_database_name }}" @@ -11,7 +20,7 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" - when: previous_build_number == 0 + when: previous_build_number == 0 or _build_db_status.rc == 1 run_once: true - name: Ensure the dump directory exists. @@ -28,3 +37,5 @@ executable: /bin/bash when: previous_build_number > 0 run_once: true + args: + executable: /bin/bash diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml index dea8fe9f..8447b3ee 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml @@ -4,6 +4,15 @@ - ansible.builtin.set_fact: _mysql_previous_build_database_name: "{{ database.database }}" +- name: Check if the database exists already. + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_build_database_name }}" + register: _build_db_status + failed_when: _build_db_status.rc > 1 # only exit on abnormal errors + run_once: true + args: + executable: /bin/bash + +# Create database if this is an initial build or it doesn't exist yet - name: Create initial database. community.mysql.mysql_db: name: "{{ _mysql_build_database_name }}" @@ -11,5 +20,5 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" - when: previous_build_number == 0 + when: previous_build_number == 0 or _build_db_status.rc == 1 run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index 5a82a644..094da9fc 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -5,7 +5,7 @@ - ansible.builtin.set_fact: _mysql_previous_build_database_name: "{{ database.database }}_{{ previous_build_number }}" -- name: Check if the new database name exists already. +- name: Check if the new database exists already. ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_build_database_name }}" register: _build_db_status failed_when: _build_db_status.rc == 0 # we want the build to fail if the database exists diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy.yml b/roles/database_backup/database_backup-mysql/tasks/deploy.yml index f6d75658..67d460e9 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy.yml @@ -82,7 +82,7 @@ _mysql_build_password: "{{ lookup('password', '/dev/shm/{{ project_name }}_{{ build_type }}_{{ build_number }}') }}" when: mysql_backup.credentials_handling == 'rotate' -- name: Set database handling type. +- name: "Set database handling type to {{ mysql_backup.handling }}." ansible.builtin.set_fact: _mysql_handling: "{{ mysql_backup.handling }}" # If we have a specific instruction for handling this database differently, use it. From e9ee0ccc3b716f1bad605a52de02106a5ddf2cf7 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 15:10:56 +0200 Subject: [PATCH 059/190] Improved database handling pr devel (#362) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup/database_backup-mysql/tasks/deploy-dump.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml index 3bd7e69e..c07a922f 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml @@ -37,5 +37,3 @@ executable: /bin/bash when: previous_build_number > 0 run_once: true - args: - executable: /bin/bash From 35c46f8dc6c01e666fef70dcad41f86757aa03b0 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 15:27:01 +0200 Subject: [PATCH 060/190] Improved database handling pr devel (#363) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup/database_backup-mysql/tasks/deploy.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy.yml b/roles/database_backup/database_backup-mysql/tasks/deploy.yml index 67d460e9..4304c3e4 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy.yml @@ -88,10 +88,10 @@ # If we have a specific instruction for handling this database differently, use it. - name: Override database handling type for this database, if specified. ansible.builtin.set_fact: - _mysql_handling: "{{ mysql_backup.database.handling }}" + _mysql_handling: "{{ database.handling }}" when: - - mysql_backup.database.handling is defined - - mysql_backup.database.handling | length > 0 + - database.handling is defined + - database.handling | length > 0 - name: Execute backup tasks. ansible.builtin.include_tasks: "deploy-{{ _mysql_handling }}.yml" From 5e84db0170d52c50c597aa87b6fcd18aa5690d62 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 15:35:12 +0200 Subject: [PATCH 061/190] Improved database handling pr devel (#364) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../database_backup-mysql/tasks/deploy-dump.yml | 2 +- .../database_backup-mysql/tasks/deploy-none.yml | 2 +- .../database_backup-mysql/tasks/deploy-rolling.yml | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml index c07a922f..49bda27e 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml @@ -5,7 +5,7 @@ _mysql_previous_build_database_name: "{{ database.database }}" - name: Check if the database exists already. - ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_build_database_name }}" + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_build_database_name }}" register: _build_db_status failed_when: _build_db_status.rc > 1 # only exit on abnormal errors run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml index 8447b3ee..4c4b1624 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml @@ -5,7 +5,7 @@ _mysql_previous_build_database_name: "{{ database.database }}" - name: Check if the database exists already. - ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_build_database_name }}" + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_build_database_name }}" register: _build_db_status failed_when: _build_db_status.rc > 1 # only exit on abnormal errors run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index 094da9fc..30fbaa62 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -6,7 +6,7 @@ _mysql_previous_build_database_name: "{{ database.database }}_{{ previous_build_number }}" - name: Check if the new database exists already. - ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_build_database_name }}" + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_build_database_name }}" register: _build_db_status failed_when: _build_db_status.rc == 0 # we want the build to fail if the database exists run_once: true @@ -22,7 +22,7 @@ run_once: true - name: Check if the previous database exists. - ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep {{ _mysql_previous_build_database_name }}" + ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_previous_build_database_name }}" register: _previous_db_status failed_when: _previous_db_status.rc > 1 # only fail if we get an unexpected exitcode run_once: true From 23c231a64f9b075b37179a5d1f4d5359af97c9be Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 22 Sep 2023 17:52:32 +0200 Subject: [PATCH 062/190] Bug fixes pr devel (#365) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../cache_clear-opcache/defaults/main.yml | 4 ++-- .../database_apply-drupal8/tasks/main.yml | 15 +++++++++++++++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/roles/cache_clear/cache_clear-opcache/defaults/main.yml b/roles/cache_clear/cache_clear-opcache/defaults/main.yml index 94e1cb22..5ff1b5c2 100644 --- a/roles/cache_clear/cache_clear-opcache/defaults/main.yml +++ b/roles/cache_clear/cache_clear-opcache/defaults/main.yml @@ -7,6 +7,6 @@ cache_clear_opcache: # adapter: "--fcgi=127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. # Bins to clear. clear_opcache: true - clear_apcu: false - clear_stat: false + clear_apcu: true + clear_stat: true # cachetool_bin: "/path/to/cachetool.phar" # see _init for paths if undefined diff --git a/roles/database_apply/database_apply-drupal8/tasks/main.yml b/roles/database_apply/database_apply-drupal8/tasks/main.yml index 7faa03c7..1935ac79 100644 --- a/roles/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/database_apply/database_apply-drupal8/tasks/main.yml @@ -13,6 +13,21 @@ - www_user != deploy_user - previous_build_number == 0 +- name: Fix file permissions for config directory. + ansible.builtin.file: + state: directory + path: "{{ deploy_path }}/{{ site.config_sync_directory }}" + owner: "{{ www_user }}" + group: "{{ www_user }}" + mode: '0775' # in this case often our deploy user is in the web user group and will need to be able to manipulate config + become: true + with_items: "{{ drupal.sites }}" + loop_control: + loop_var: site + when: + - www_user != deploy_user + - previous_build_number == 0 + - name: Install Drupal. ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site.folder }} {{ site.install_command }}" From 69027e2340902146649eeeadbd8f720eac002c81 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 25 Sep 2023 10:36:08 +0200 Subject: [PATCH 063/190] Bug fixes pr devel (#367) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/cache_clear/cache_clear-opcache/defaults/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/cache_clear/cache_clear-opcache/defaults/main.yml b/roles/cache_clear/cache_clear-opcache/defaults/main.yml index 5ff1b5c2..d7f4af62 100644 --- a/roles/cache_clear/cache_clear-opcache/defaults/main.yml +++ b/roles/cache_clear/cache_clear-opcache/defaults/main.yml @@ -7,6 +7,6 @@ cache_clear_opcache: # adapter: "--fcgi=127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. # Bins to clear. clear_opcache: true - clear_apcu: true + clear_apcu: false clear_stat: true # cachetool_bin: "/path/to/cachetool.phar" # see _init for paths if undefined From 6d442eebaa4f361f5cc3bceac6203eccb2648836 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 25 Sep 2023 13:30:35 +0200 Subject: [PATCH 064/190] Bug fixes pr devel (#369) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_code/tasks/cleanup.yml | 122 ++++++++++++++-------------- 1 file changed, 61 insertions(+), 61 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 870f86b7..73834d91 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -1,4 +1,18 @@ --- +- name: Build warning. + ansible.builtin.debug: + msg: "### WARNING - if you are using a server terminal exit NOW! You have 20 seconds to comply..." + run_once: true + when: + - deploy_code.mount_type == "squashfs" + +- name: Give people 20 seconds to exit terminal sessions. + ansible.builtin.pause: + seconds: 20 + run_once: true + when: + - deploy_code.mount_type == "squashfs" + - name: Ensure codebase is writable. ansible.builtin.shell: cmd: "if [ -d {{ deploy_path_prefix }}{{ item }} ]; then chmod -R 777 {{ deploy_path_prefix }}{{ item }}; fi" @@ -139,72 +153,58 @@ - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" -- name: Reload any services that might be keeping the loop device busy. - ansible.builtin.service: - name: "{{ www_service }}" - state: reloaded - with_items: "{{ deploy_code.services }}" - loop_control: - loop_var: www_service - become: true - when: - - deploy_code.service_action == "reload" - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_check.rc == 0 - - deploy_code.services | length > 0 - -- name: Stop any services that might be keeping the loop device busy. - ansible.builtin.service: - name: "{{ www_service }}" - state: stopped - with_items: "{{ deploy_code.services }}" - loop_control: - loop_var: www_service - become: true - when: - - deploy_code.service_action == "stop" - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_check.rc == 0 - - deploy_code.services | length > 0 - -- name: Unmount existing SquashFS image. - ansible.builtin.command: - cmd: "umount {{ deploy_base_path }}" - become: true +- name: Unmount and remount squashfs images. when: - deploy_code.mount_sync is defined - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" - _deploy_code_mount_check.rc == 0 - -- name: Mount new SquashFS image. - ansible.builtin.command: - cmd: "mount {{ build_base_path }}/deploy.sqsh {{ deploy_base_path }} -t squashfs -o loop" - become: true - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - -- name: Start any services we stopped. - ansible.builtin.service: - name: "{{ www_service }}" - state: started - with_items: "{{ deploy_code.services }}" - loop_control: - loop_var: www_service - become: true - when: - - deploy_code.service_action == "stop" - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_check is succeeded - - deploy_code.services | length > 0 + block: + - name: Reload any services that might be keeping the loop device busy. + ansible.builtin.service: + name: "{{ www_service }}" + state: reloaded + with_items: "{{ deploy_code.services }}" + loop_control: + loop_var: www_service + become: true + when: + - deploy_code.service_action == "reload" + - deploy_code.services | length > 0 + + - name: Stop any services that might be keeping the loop device busy. + ansible.builtin.service: + name: "{{ www_service }}" + state: stopped + with_items: "{{ deploy_code.services }}" + loop_control: + loop_var: www_service + become: true + when: + - deploy_code.service_action == "stop" + - deploy_code.services | length > 0 + + - name: Unmount existing SquashFS image. + ansible.builtin.command: + cmd: "umount {{ deploy_base_path }}" + become: true + + - name: Mount new SquashFS image. + ansible.builtin.command: + cmd: "mount {{ build_base_path }}/deploy.sqsh {{ deploy_base_path }} -t squashfs -o loop" + become: true + + - name: Start any services we stopped. + ansible.builtin.service: + name: "{{ www_service }}" + state: started + with_items: "{{ deploy_code.services }}" + loop_control: + loop_var: www_service + become: true + when: + - deploy_code.service_action == "stop" + - deploy_code.services | length > 0 - name: Trigger an infrastructure rebuild. ansible.builtin.include_role: From 910e10aa14418705233e9bc8e91645d922ed5be6 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 25 Sep 2023 17:50:25 +0200 Subject: [PATCH 065/190] Bug fixes pr devel (#370) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_code/tasks/cleanup.yml | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 73834d91..8dd737b9 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -141,25 +141,21 @@ - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" -- name: Check if we have a mount already. - ansible.builtin.shell: - cmd: "set -o pipefail && mount | grep {{ deploy_base_path }}" - args: - executable: /bin/bash - failed_when: false - register: _deploy_code_mount_check - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - - name: Unmount and remount squashfs images. when: - deploy_code.mount_sync is defined - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_check.rc == 0 + throttle: 1 # execute one machine at a time block: + - name: Check if we have a mount already. + ansible.builtin.shell: + cmd: "set -o pipefail && mount | grep {{ deploy_base_path }}" + args: + executable: /bin/bash + failed_when: false + register: _deploy_code_mount_check + - name: Reload any services that might be keeping the loop device busy. ansible.builtin.service: name: "{{ www_service }}" @@ -169,6 +165,7 @@ loop_var: www_service become: true when: + - _deploy_code_mount_check.rc == 0 - deploy_code.service_action == "reload" - deploy_code.services | length > 0 @@ -181,6 +178,7 @@ loop_var: www_service become: true when: + - _deploy_code_mount_check.rc == 0 - deploy_code.service_action == "stop" - deploy_code.services | length > 0 @@ -188,6 +186,8 @@ ansible.builtin.command: cmd: "umount {{ deploy_base_path }}" become: true + when: + - _deploy_code_mount_check.rc == 0 - name: Mount new SquashFS image. ansible.builtin.command: @@ -203,6 +203,7 @@ loop_var: www_service become: true when: + - _deploy_code_mount_check.rc == 0 - deploy_code.service_action == "stop" - deploy_code.services | length > 0 From 3bf5000b4a13a80ebd6f73694fd02c199cfa3621 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 25 Sep 2023 18:02:15 +0200 Subject: [PATCH 066/190] Bug fixes pr devel (#371) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- roles/deploy_code/tasks/cleanup.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 8dd737b9..776b0557 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -146,7 +146,6 @@ - deploy_code.mount_sync is defined - deploy_code.mount_sync | length > 1 - deploy_code.mount_type == "squashfs" - throttle: 1 # execute one machine at a time block: - name: Check if we have a mount already. ansible.builtin.shell: From ca09eefb57664150d0285edbb66b655852865f13 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 27 Sep 2023 13:50:58 +0200 Subject: [PATCH 067/190] Bug fixes pr devel (#373) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> --- .../config_generate-drupal8/tasks/settings.yml | 4 ++++ .../config_generate-drupal8/templates/settings.php.j2 | 2 ++ 2 files changed, 6 insertions(+) diff --git a/roles/config_generate/config_generate-drupal8/tasks/settings.yml b/roles/config_generate/config_generate-drupal8/tasks/settings.yml index 6f698ba5..79e0ad5e 100644 --- a/roles/config_generate/config_generate-drupal8/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal8/tasks/settings.yml @@ -18,6 +18,10 @@ mode: 0775 become: "{{ false if www_user == deploy_user else true }}" +- name: Generate or retrieve hash salt value. + ansible.builtin.set_fact: + _drupal_hash_salt: "{{ lookup('ansible.builtin.password', _ce_deploy_data_dir + '/' + project_name + '_' + build_type + '/drupal/hash_salt length=74 chars=ascii_letters,digits') }}" + - name: Generates settings.php file. ansible.builtin.template: src: "{{ item }}" diff --git a/roles/config_generate/config_generate-drupal8/templates/settings.php.j2 b/roles/config_generate/config_generate-drupal8/templates/settings.php.j2 index e5d43204..895f654f 100644 --- a/roles/config_generate/config_generate-drupal8/templates/settings.php.j2 +++ b/roles/config_generate/config_generate-drupal8/templates/settings.php.j2 @@ -23,3 +23,5 @@ $settings['file_public_path'] = '{{ build_public_file_path }}'; $config_directories['sync'] = '{{ build_config_sync_directory }}'; // Drupal 8.8 $settings['config_sync_directory'] = '{{ build_config_sync_directory }}'; + +$settings['hash_salt'] = '{{ _drupal_hash_salt }}'; From 33f4b64dd352649f701fb39f3c82cbcd0f4379b9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matej=20=C5=A0tajduhar?= <30931414+matej5@users.noreply.github.com> Date: Wed, 4 Oct 2023 18:09:14 +0200 Subject: [PATCH 068/190] R67016 changing path for database backup task pr devel (#355) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * r67016-changing-path-for-database_backup-task * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * Removing rogue extra line. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- docs/roles/cache_clear/cache_clear-opcache.md | 2 +- docs/roles/database_backup/database_backup-mysql.md | 3 +++ roles/cache_clear/cache_clear-opcache/README.md | 2 +- roles/database_backup/database_backup-mysql/README.md | 3 +++ .../database_backup-mysql/defaults/main.yml | 2 ++ .../database_backup/database_backup-mysql/tasks/deploy.yml | 7 ++++++- 6 files changed, 16 insertions(+), 3 deletions(-) diff --git a/docs/roles/cache_clear/cache_clear-opcache.md b/docs/roles/cache_clear/cache_clear-opcache.md index 465307e2..bb0bd05d 100644 --- a/docs/roles/cache_clear/cache_clear-opcache.md +++ b/docs/roles/cache_clear/cache_clear-opcache.md @@ -18,7 +18,7 @@ cache_clear_opcache: # Bins to clear. clear_opcache: true clear_apcu: false - clear_stat: false + clear_stat: true # cachetool_bin: "/path/to/cachetool.phar" # see _init for paths if undefined ``` diff --git a/docs/roles/database_backup/database_backup-mysql.md b/docs/roles/database_backup/database_backup-mysql.md index 3a7ec471..6637e2da 100644 --- a/docs/roles/database_backup/database_backup-mysql.md +++ b/docs/roles/database_backup/database_backup-mysql.md @@ -8,6 +8,8 @@ mysql_backup: handling: rolling dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/build" mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here + # Location on deploy server where the generated MySQL password will be stashed - should be temporary storage + mysql_password_path: "/tmp/.ce-deploy/{{ project_name }}_{{ build_type }}_{{ build_number }}" # Number of dumps/db to keep. Note this is independant from the build codebases. keep: 10 # This can be one of the following: @@ -27,6 +29,7 @@ mysql_backup: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" #handling: static # optional override to the main handling method on a per database basis + ``` diff --git a/roles/cache_clear/cache_clear-opcache/README.md b/roles/cache_clear/cache_clear-opcache/README.md index 465307e2..bb0bd05d 100644 --- a/roles/cache_clear/cache_clear-opcache/README.md +++ b/roles/cache_clear/cache_clear-opcache/README.md @@ -18,7 +18,7 @@ cache_clear_opcache: # Bins to clear. clear_opcache: true clear_apcu: false - clear_stat: false + clear_stat: true # cachetool_bin: "/path/to/cachetool.phar" # see _init for paths if undefined ``` diff --git a/roles/database_backup/database_backup-mysql/README.md b/roles/database_backup/database_backup-mysql/README.md index 3a7ec471..6637e2da 100644 --- a/roles/database_backup/database_backup-mysql/README.md +++ b/roles/database_backup/database_backup-mysql/README.md @@ -8,6 +8,8 @@ mysql_backup: handling: rolling dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/build" mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here + # Location on deploy server where the generated MySQL password will be stashed - should be temporary storage + mysql_password_path: "/tmp/.ce-deploy/{{ project_name }}_{{ build_type }}_{{ build_number }}" # Number of dumps/db to keep. Note this is independant from the build codebases. keep: 10 # This can be one of the following: @@ -27,6 +29,7 @@ mysql_backup: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" #handling: static # optional override to the main handling method on a per database basis + ``` diff --git a/roles/database_backup/database_backup-mysql/defaults/main.yml b/roles/database_backup/database_backup-mysql/defaults/main.yml index 777cc6f9..1091f8d7 100644 --- a/roles/database_backup/database_backup-mysql/defaults/main.yml +++ b/roles/database_backup/database_backup-mysql/defaults/main.yml @@ -3,6 +3,8 @@ mysql_backup: handling: rolling dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/build" mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here + # Location on deploy server where the generated MySQL password will be stashed - should be temporary storage + mysql_password_path: "/tmp/.ce-deploy/{{ project_name }}_{{ build_type }}_{{ build_number }}" # Number of dumps/db to keep. Note this is independant from the build codebases. keep: 10 # This can be one of the following: diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy.yml b/roles/database_backup/database_backup-mysql/tasks/deploy.yml index 4304c3e4..8d93c118 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy.yml @@ -71,6 +71,11 @@ - _legacy_static_creds.stat.exists ### End of legacy handling. +- name: Ensure the temporary password directory exists. + ansible.builtin.file: + state: directory + path: "{{ mysql_backup.mysql_password_path }}" + - ansible.builtin.set_fact: _mysql_build_password: "{{ lookup('password', '{{ _ce_deploy_data_dir }}/{{ project_name }}_{{ build_type }}/mysql/{{ _mysql_host }}/{{ database.database }}') }}" when: mysql_backup.credentials_handling == 'static' @@ -79,7 +84,7 @@ _mysql_build_user_name: "{{ database.user }}_{{ build_number }}" when: mysql_backup.credentials_handling == 'rotate' - ansible.builtin.set_fact: - _mysql_build_password: "{{ lookup('password', '/dev/shm/{{ project_name }}_{{ build_type }}_{{ build_number }}') }}" + _mysql_build_password: "{{ lookup('password', '{{ mysql_backup.mysql_password_path }}') }}" when: mysql_backup.credentials_handling == 'rotate' - name: "Set database handling type to {{ mysql_backup.handling }}." From 001bba7816a9b6c4d1eb019c764c4cf77cc69550 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 31 Oct 2023 17:34:48 +0100 Subject: [PATCH 069/190] 67412 callback plugin fail no hosts pr devel (#376) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- plugins/callback/fail_on_no_hosts.py | 30 +++++ plugins/vars/sops_vars.py | 163 --------------------------- 2 files changed, 30 insertions(+), 163 deletions(-) create mode 100644 plugins/callback/fail_on_no_hosts.py delete mode 100644 plugins/vars/sops_vars.py diff --git a/plugins/callback/fail_on_no_hosts.py b/plugins/callback/fail_on_no_hosts.py new file mode 100644 index 00000000..862ae9a4 --- /dev/null +++ b/plugins/callback/fail_on_no_hosts.py @@ -0,0 +1,30 @@ +import sys + +from ansible.plugins.callback import CallbackBase + +DOCUMENTATION = ''' +name: fail_on_no_hosts +callback_type: aggregate +requirements: + - enable in configuration +short_description: Exits with code 1 if no play hosts are matched +version_added: "2.0" +description: + - This callback overrides the default 'v2_playbook_on_no_hosts_matched' method with one that exits instead of just notifying. +''' + +class CallbackModule(CallbackBase): + """ + This callback module exists non-zero if no hosts match + """ + CALLBACK_VERSION = 2.0 + CALLBACK_TYPE = 'aggregate' + CALLBACK_NAME = 'fail_on_no_hosts' + CALLBACK_NEEDS_WHITELIST = False + + def __init__(self): + super(CallbackModule, self).__init__() + + def v2_playbook_on_no_hosts_matched(self): + self._display.display("failed: no hosts matched", color=C.COLOR_ERROR) + sys.exit(1) diff --git a/plugins/vars/sops_vars.py b/plugins/vars/sops_vars.py deleted file mode 100644 index eca3f3b9..00000000 --- a/plugins/vars/sops_vars.py +++ /dev/null @@ -1,163 +0,0 @@ -# -*- coding: utf-8 -*- - -# Copyright: (c) 2019, Arduino, srl -# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) -# -############################################# - -from __future__ import (absolute_import, division, print_function) -__metaclass__ = type - -DOCUMENTATION = ''' - vars: sops_vars - author: Edoardo Tenani (@endorama) - version_added: "2.10" - short_description: Loading sops-encrypted vars files - description: - - Load encrypted YAML files into correspondind groups/hosts in group_vars/ and host_vars/ directories. - - Files are encrypted prior to reading, making this plugin an effective companion to host_group_vars plugin. - - Files are restricted to .sops.yaml, .sops.yml, .sops.json extensions. - - Hidden files are ignored. - options: - _valid_extensions: - default: [".sops.yml", ".sops.yaml", ".sops.json"] - description: - - "Check all of these extensions when looking for 'variable' files which should be YAML or JSON or vaulted versions of these." - - 'This affects vars_files, include_vars, inventory and vars plugins among others.' - type: list -''' - -import os -from ansible import constants as C -from ansible.errors import AnsibleParserError -from ansible.module_utils._text import to_bytes, to_native, to_text -from ansible.plugins.vars import BaseVarsPlugin -from ansible.inventory.host import Host -from ansible.inventory.group import Group -from ansible.utils.vars import combine_vars -from ansible.errors import AnsibleError -from subprocess import Popen, PIPE -from ansible.utils.display import Display -display = Display() - -FOUND = {} -DEFAULT_VALID_EXTENSIONS = [".sops.yaml", ".sops.yml", ".sops.json"] - -# From https://github.com/mozilla/sops/blob/master/cmd/sops/codes/codes.go -# Should be manually updated -sops_error_codes = { - 1: "SopsErrorGeneric", - 2: "SopsCouldNotReadInputFile", - 3: "SopsCouldNotWriteOutputFile", - 4: "SopsErrorDumpingTree", - 5: "SopsErrorReadingConfig", - 6: "SopsErrorInvalidKMSEncryptionContextFormat", - 7: "SopsErrorInvalidSetFormat", - 8: "SopsErrorConflictingParameters", - 21: "SopsErrorEncryptingMac", - 23: "SopsErrorEncryptingTree", - 24: "SopsErrorDecryptingMac", - 25: "SopsErrorDecryptingTree", - 49: "SopsCannotChangeKeysFromNonExistentFile", - 51: "SopsMacMismatch", - 52: "SopsMacNotFound", - 61: "SopsConfigFileNotFound", - 85: "SopsKeyboardInterrupt", - 91: "SopsInvalidTreePathFormat", - 100: "SopsNoFileSpecified", - 128: "SopsCouldNotRetrieveKey", - 111: "SopsNoEncryptionKeyFound", - 200: "SopsFileHasNotBeenModified", - 201: "SopsNoEditorFound", - 202: "SopsFailedToCompareVersions", - 203: "SopsFileAlreadyEncrypted" -} - - -class SopsError(AnsibleError): - ''' extend AnsibleError class with sops specific informations ''' - - def __init__(self, filename, exit_code, message,): - exception_name = sops_error_codes[exit_code] - message = "error with file %s: %s exited with code %d: %s" % (filename, exception_name, exit_code, message) - super(SopsError, self).__init__(message=message) - - -def decrypt_with_sops(filename): - display.vvvv(u"sops --decrypt %s" % filename) - - # Run sops directly as python module is deprecated - process = Popen(["sops", "--decrypt", filename], stdout=PIPE, stderr=PIPE) - (output, err) = process.communicate() - exit_code = process.wait() - - # DO NOT display output - # is the decrypted secret and would easily end in logs :) - # if output: - # display.vvvv(output) - - # sops logs always to stderr ( stdout is used for file content ) - if err: - display.vvvv(err) - - if exit_code > 0: - if exit_code in sops_error_codes.keys(): - raise SopsError(filename, exit_code, err) - else: - raise AnsibleError(message=err) - - return output - - -class VarsModule(BaseVarsPlugin): - - def get_vars(self, loader, path, entities, cache=True): - ''' parses the inventory file ''' - - if not isinstance(entities, list): - entities = [entities] - - super(VarsModule, self).get_vars(loader, path, entities) - - data = {} - for entity in entities: - if isinstance(entity, Host): - subdir = 'host_vars' - elif isinstance(entity, Group): - subdir = 'group_vars' - else: - raise AnsibleParserError("Supplied entity must be Host or Group, got %s instead" % (type(entity))) - - # avoid 'chroot' type inventory hostnames /path/to/chroot - if not entity.name.startswith(os.path.sep): - try: - found_files = [] - # load vars - b_opath = os.path.realpath(to_bytes(os.path.join(self._basedir, subdir))) - opath = to_text(b_opath) - key = '%s.%s' % (entity.name, opath) - self._display.vvvv("key: %s" % (key)) - if cache and key in FOUND: - found_files = FOUND[key] - else: - # no need to do much if path does not exist for basedir - if os.path.exists(b_opath): - if os.path.isdir(b_opath): - self._display.debug("\tprocessing dir %s" % opath) - found_files = loader.find_vars_files(opath, entity.name) - found_files = [file_path for file_path in found_files - if any(file_path.endswith(extension) for extension in DEFAULT_VALID_EXTENSIONS)] - FOUND[key] = found_files - else: - self._display.warning("Found %s that is not a directory, skipping: %s" % (subdir, opath)) - - for found in found_files: - file_content = decrypt_with_sops(found) - new_data = loader.load(file_content) - if new_data: # ignore empty files - data = combine_vars(data, new_data) - - except Exception as e: - raise AnsibleParserError(to_native(e)) - - return data From ba1d7fa105c41f5d0e1509edbb1f9dc21b263e9c Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 31 Oct 2023 18:21:37 +0100 Subject: [PATCH 070/190] Bug fixes pr devel (#378) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- plugins/callback/fail_on_no_hosts.py | 1 + 1 file changed, 1 insertion(+) diff --git a/plugins/callback/fail_on_no_hosts.py b/plugins/callback/fail_on_no_hosts.py index 862ae9a4..e022c0a4 100644 --- a/plugins/callback/fail_on_no_hosts.py +++ b/plugins/callback/fail_on_no_hosts.py @@ -1,5 +1,6 @@ import sys +from ansible import constants as C from ansible.plugins.callback import CallbackBase DOCUMENTATION = ''' From 458ced09480697f847c152fc551526dfd0c6caa9 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 2 Nov 2023 12:58:59 +0100 Subject: [PATCH 071/190] Bug fixes pr devel (#380) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- plugins/README.md | 16 ++++ plugins/vars/sops_vars.py | 163 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 179 insertions(+) create mode 100644 plugins/README.md create mode 100644 plugins/vars/sops_vars.py diff --git a/plugins/README.md b/plugins/README.md new file mode 100644 index 00000000..58e31070 --- /dev/null +++ b/plugins/README.md @@ -0,0 +1,16 @@ +# Ansible Plugins +This directory contains extra plugins for Ansible. + +## Vars plugins +Here we include a plugin for handling SOPS decryption. + +## Callback plugins +Here we have a small custom override that fails builds if there are no matching hosts found, to avoid `ce-deploy` incrementing the track file when it didn't actually run because of a host issue. + +## Enabling plugins +To use these plugins you need to find the `# set plugin path directories here` section of `ansible.cfg` which should be kept in your `ce-deploy-config` repository. Add the paths to the plugin directories to enable the plugins, e.g. + +``` +vars_plugins = /home/deploy/ce-deploy/plugins/vars:/usr/share/ansible/plugins/vars +callback_plugins = /home/deploy/ce-deploy/plugins/callback:/usr/share/ansible/plugins/callback +``` diff --git a/plugins/vars/sops_vars.py b/plugins/vars/sops_vars.py new file mode 100644 index 00000000..9193d61c --- /dev/null +++ b/plugins/vars/sops_vars.py @@ -0,0 +1,163 @@ +# -*- coding: utf-8 -*- + +# Copyright: (c) 2019, Arduino, srl +# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) +# +############################################# + +from __future__ import (absolute_import, division, print_function) +__metaclass__ = type + +DOCUMENTATION = ''' + vars: sops_vars + author: Edoardo Tenani (@endorama) + version_added: "2.10" + short_description: Loading sops-encrypted vars files + description: + - Load encrypted YAML files into correspondind groups/hosts in group_vars/ and host_vars/ directories. + - Files are encrypted prior to reading, making this plugin an effective companion to host_group_vars plugin. + - Files are restricted to .sops.yaml, .sops.yml, .sops.json extensions. + - Hidden files are ignored. + options: + _valid_extensions: + default: [".sops.yml", ".sops.yaml", ".sops.json"] + description: + - "Check all of these extensions when looking for 'variable' files which should be YAML or JSON or vaulted versions of these." + - 'This affects vars_files, include_vars, inventory and vars plugins among others.' + type: list +''' + +import os +from ansible import constants as C +from ansible.errors import AnsibleParserError +from ansible.module_utils._text import to_bytes, to_native, to_text +from ansible.plugins.vars import BaseVarsPlugin +from ansible.inventory.host import Host +from ansible.inventory.group import Group +from ansible.utils.vars import combine_vars +from ansible.errors import AnsibleError +from subprocess import Popen, PIPE +from ansible.utils.display import Display +display = Display() + +FOUND = {} +DEFAULT_VALID_EXTENSIONS = [".sops.yaml", ".sops.yml", ".sops.json"] + +# From https://github.com/mozilla/sops/blob/master/cmd/sops/codes/codes.go +# Should be manually updated +sops_error_codes = { + 1: "SopsErrorGeneric", + 2: "SopsCouldNotReadInputFile", + 3: "SopsCouldNotWriteOutputFile", + 4: "SopsErrorDumpingTree", + 5: "SopsErrorReadingConfig", + 6: "SopsErrorInvalidKMSEncryptionContextFormat", + 7: "SopsErrorInvalidSetFormat", + 8: "SopsErrorConflictingParameters", + 21: "SopsErrorEncryptingMac", + 23: "SopsErrorEncryptingTree", + 24: "SopsErrorDecryptingMac", + 25: "SopsErrorDecryptingTree", + 49: "SopsCannotChangeKeysFromNonExistentFile", + 51: "SopsMacMismatch", + 52: "SopsMacNotFound", + 61: "SopsConfigFileNotFound", + 85: "SopsKeyboardInterrupt", + 91: "SopsInvalidTreePathFormat", + 100: "SopsNoFileSpecified", + 128: "SopsCouldNotRetrieveKey", + 111: "SopsNoEncryptionKeyFound", + 200: "SopsFileHasNotBeenModified", + 201: "SopsNoEditorFound", + 202: "SopsFailedToCompareVersions", + 203: "SopsFileAlreadyEncrypted" +} + + +class SopsError(AnsibleError): + ''' extend AnsibleError class with sops specific informations ''' + + def __init__(self, filename, exit_code, message,): + exception_name = sops_error_codes[exit_code] + message = "error with file %s: %s exited with code %d: %s" % (filename, exception_name, exit_code, message) + super(SopsError, self).__init__(message=message) + + +def decrypt_with_sops(filename): + display.vvvv(u"sops --decrypt %s" % filename) + + # Run sops directly as python module is deprecated + process = Popen(["sops", "--decrypt", filename], stdout=PIPE, stderr=PIPE) + (output, err) = process.communicate() + exit_code = process.wait() + + # DO NOT display output + # is the decrypted secret and would easily end in logs :) + # if output: + # display.vvvv(output) + + # sops logs always to stderr ( stdout is used for file content ) + if err: + display.vvvv(err) + + if exit_code > 0: + if exit_code in sops_error_codes.keys(): + raise SopsError(filename, exit_code, err) + else: + raise AnsibleError(message=err) + + return output + + +class VarsModule(BaseVarsPlugin): + + def get_vars(self, loader, path, entities, cache=True): + ''' parses the inventory file ''' + + if not isinstance(entities, list): + entities = [entities] + + super(VarsModule, self).get_vars(loader, path, entities) + + data = {} + for entity in entities: + if isinstance(entity, Host): + subdir = 'host_vars' + elif isinstance(entity, Group): + subdir = 'group_vars' + else: + raise AnsibleParserError("Supplied entity must be Host or Group, got %s instead" % (type(entity))) + + # avoid 'chroot' type inventory hostnames /path/to/chroot + if not entity.name.startswith(os.path.sep): + try: + found_files = [] + # load vars + b_opath = os.path.realpath(to_bytes(os.path.join(self._basedir, subdir))) + opath = to_text(b_opath) + key = '%s.%s' % (entity.name, opath) + self._display.vvvv("key: %s" % (key)) + if cache and key in FOUND: + found_files = FOUND[key] + else: + # no need to do much if path does not exist for basedir + if os.path.exists(b_opath): + if os.path.isdir(b_opath): + self._display.debug("\tprocessing dir %s" % opath) + found_files = loader.find_vars_files(opath, entity.name) + found_files = [file_path for file_path in found_files + if any(file_path.endswith(extension) for extension in DEFAULT_VALID_EXTENSIONS)] + FOUND[key] = found_files + else: + self._display.warning("Found %s that is not a directory, skipping: %s" % (subdir, opath)) + + for found in found_files: + file_content = decrypt_with_sops(found) + new_data = loader.load(file_content) + if new_data: # ignore empty files + data = combine_vars(data, new_data) + + except Exception as e: + raise AnsibleParserError(to_native(e)) + + return data \ No newline at end of file From dcaf18c6fcdb827e6e992e75beb2807f8eeadc00 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 7 Nov 2023 16:43:45 +0100 Subject: [PATCH 072/190] Bug fixes pr devel (#382) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/drupal7.yml | 8 ++++++-- roles/_init/tasks/drupal8.yml | 6 +++++- .../maintenance_mode-drupal-core/tasks/offline.yml | 4 ++-- .../maintenance_mode-drupal-core/tasks/online.yml | 8 ++++---- 4 files changed, 17 insertions(+), 9 deletions(-) diff --git a/roles/_init/tasks/drupal7.yml b/roles/_init/tasks/drupal7.yml index 133b3d73..b1781c1b 100644 --- a/roles/_init/tasks/drupal7.yml +++ b/roles/_init/tasks/drupal7.yml @@ -1,8 +1,12 @@ --- -- name: Define Drush path. +- name: Define path to drush for this build. set_fact: drush_bin: "{{ drush_bin | default('{{ bin_directory }}/drush.phar') }}" -- name: Ensure we have Drush binary. +- name: Define path to drush for currently live build. # usually the same as drush_bin for Drupal 7 + set_fact: + drush_live_bin: "{{ drush_live_bin | default('{{ bin_directory }}/drush.phar') }}" + +- name: Ensure we have a drush binary. import_role: name: cli/drush diff --git a/roles/_init/tasks/drupal8.yml b/roles/_init/tasks/drupal8.yml index ebe63b82..3792c489 100644 --- a/roles/_init/tasks/drupal8.yml +++ b/roles/_init/tasks/drupal8.yml @@ -1,4 +1,8 @@ --- -- name: Define Drush path. +- name: Define path to drush for this build. set_fact: drush_bin: "{{ drush_bin | default('{{ deploy_path }}/vendor/bin/drush') }}" + +- name: Define path to drush for currently live build. + set_fact: + drush_live_bin: "{{ drush_live_bin | default('{{ live_symlink_dest }}/vendor/bin/drush') }}" \ No newline at end of file diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml index 2d6a9310..78314024 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml @@ -1,7 +1,7 @@ --- - name: Enable maintenance mode. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 1 --input-format=integer --root {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_live_bin }} -l {{ site.folder }} state:set system.maintenance_mode 1 --input-format=integer --root {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" args: chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" @@ -13,7 +13,7 @@ # For some reason D7 drush doesn't respect 'chdir' with command, using shell instead. - name: Enable maintenance mode D7. ansible.builtin.shell: - cmd: "{{ drush_bin }} -l {{ site.folder }} vset maintenance_mode 1" + cmd: "{{ drush_live_bin }} -l {{ site.folder }} vset maintenance_mode 1" chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml index 6b127f89..2598481a 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml @@ -1,9 +1,9 @@ --- - name: Disable maintenance mode. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_live_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" args: - chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: @@ -12,8 +12,8 @@ # For some reason D7 drush doesn't respect 'chdir' with command, using shell instead. - name: Disable maintenance mode D7. ansible.builtin.shell: - cmd: "{{ drush_bin }} -l {{ site.folder }} vset maintenance_mode 0" - chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_live_bin }} -l {{ site.folder }} vset maintenance_mode 0" + chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: From 6b26f20e6dde25f053a02dd0fc0d4ff691bc5f4e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 9 Nov 2023 10:30:25 +0100 Subject: [PATCH 073/190] Bug fixes pr devel (#385) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../maintenance_mode-drupal-core/tasks/online.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml index 2598481a..6b127f89 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml @@ -1,9 +1,9 @@ --- - name: Disable maintenance mode. ansible.builtin.command: - cmd: "{{ drush_live_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" args: - chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" + chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: @@ -12,8 +12,8 @@ # For some reason D7 drush doesn't respect 'chdir' with command, using shell instead. - name: Disable maintenance mode D7. ansible.builtin.shell: - cmd: "{{ drush_live_bin }} -l {{ site.folder }} vset maintenance_mode 0" - chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_bin }} -l {{ site.folder }} vset maintenance_mode 0" + chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: From fdd8bca05bdd46388dc409d86b20759bfc5566cc Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 13 Nov 2023 17:06:03 +0100 Subject: [PATCH 074/190] Bug fixes pr devel (#387) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- scripts/_common.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/scripts/_common.sh b/scripts/_common.sh index 74ba9a60..7c7c7659 100755 --- a/scripts/_common.sh +++ b/scripts/_common.sh @@ -21,19 +21,26 @@ BUILD_WORKSPACE_BASE="$OWN_DIR/build" DRY_RUN="no" VERBOSE="no" BOTO_PROFILE="" +# Ensure build workspace exists. if [ ! -d "$BUILD_WORKSPACE_BASE" ]; then mkdir "$BUILD_WORKSPACE_BASE" fi BUILD_TMP_DIR=$(mktemp -d -p "$BUILD_WORKSPACE_BASE") +# Ensure ce-deploy data directory exists. ANSIBLE_DATA_DIR="$OWN_DIR/data" if [ ! -d "$ANSIBLE_DATA_DIR" ]; then mkdir "$ANSIBLE_DATA_DIR" fi +# Ensure directory for build track files exists. BUILD_TRACK_DIR="$OWN_DIR/track" if [ ! -d "$BUILD_TRACK_DIR" ]; then mkdir "$BUILD_TRACK_DIR" fi ANSIBLE_LOCATION=$(command -v ansible) +# Load the contents of profile.d in case we added items to $PATH there. +for f in /etc/profile.d/*; do + . "$f" +done # Parse options arguments. parse_options(){ while [ "${1:-}" ]; do From 720a72c76df1146738365e856ccf14b15abbb4cd Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 13 Nov 2023 17:22:21 +0100 Subject: [PATCH 075/190] Bug fixes pr devel (#389) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- scripts/_common.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/_common.sh b/scripts/_common.sh index 7c7c7659..04e14c18 100755 --- a/scripts/_common.sh +++ b/scripts/_common.sh @@ -39,6 +39,7 @@ fi ANSIBLE_LOCATION=$(command -v ansible) # Load the contents of profile.d in case we added items to $PATH there. for f in /etc/profile.d/*; do +# shellcheck source=/dev/null . "$f" done # Parse options arguments. From bfd231db85e9927fac0d51639558ef7ba81cd471 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 16 Nov 2023 17:34:31 +0100 Subject: [PATCH 076/190] Bug fixes pr devel (#390) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_container/defaults/main.yml | 2 +- .../cache_clear-drupal8/tasks/main.yml | 16 ++-------------- .../database_apply-drupal8/tasks/main.yml | 18 +++--------------- 3 files changed, 6 insertions(+), 30 deletions(-) diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index be178aca..f01dcb15 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -68,7 +68,7 @@ deploy_container: launch_type: FARGATE network_mode: awsvpc #volumes: [] # list of additional volumes to attach - target_group_name: example # 32 character limit + target_group_name: example # can have a maximum of 32 characters, must contain only alphanumeric characters or hyphens, and must not begin or end with a hyphen target_group_protocol: http target_group_port: 8080 # ports lower than 1024 will require the app to be configured to run as a privileged user in the Dockerfile target_group_wait_timeout: 200 # how long to wait for target group events to complete diff --git a/roles/sync/drupal_sync_tasks/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/sync/drupal_sync_tasks/cache_clear/cache_clear-drupal8/tasks/main.yml index 431366cf..0f116adc 100644 --- a/roles/sync/drupal_sync_tasks/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/sync/drupal_sync_tasks/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,16 +1,8 @@ --- -# Drupal 8 ships drush with the website code so we need the previous build in the path. -- name: Stash the drush_bin variable. - ansible.builtin.set_fact: - _drush_bin_deploy: "{{ drush_bin }}" - -- name: Update location of drush for the sync cache clear command. - ansible.builtin.set_fact: - drush_bin: "{{ live_symlink_dest }}/vendor/bin/drush" - +# Drupal 8 ships drush with the website code so to use the drush_live_bin variable - name: Clear Drupal cache. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} -y cr" + cmd: "{{ drush_live_bin }} -l {{ site.folder }} -y cr" chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" @@ -18,7 +10,3 @@ loop_control: loop_var: site run_once: true - -- name: Restore the drush_bin variable. - ansible.builtin.set_fact: - drush_bin: "{{ _drush_bin_deploy }}" diff --git a/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml b/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml index d13b0129..297e476b 100644 --- a/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml +++ b/roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal8/tasks/main.yml @@ -1,16 +1,8 @@ --- -# Drupal 8 ships drush with the website code so we need the previous build in the path. -- name: Stash the drush_bin variable. - ansible.builtin.set_fact: - _drush_bin_deploy: "{{ drush_bin }}" - -- name: Update location of drush for the update/config import commands. - ansible.builtin.set_fact: - drush_bin: "{{ live_symlink_dest }}/vendor/bin/drush" - +# Drupal 8 ships drush with the website code so to use the drush_live_bin variable - name: Apply Drupal database updates. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} -y updb" + cmd: "{{ drush_live_bin }} -l {{ site.folder }} -y updb" chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" @@ -27,7 +19,7 @@ # This only runs if the sync_config_import variable is not defined or it is defined and is true. If it's defined and false, this won't run. - name: Import configuration. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} -y {{ site.config_import_command }}" + cmd: "{{ drush_live_bin }} -l {{ site.folder }} -y {{ site.config_import_command }}" chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" @@ -47,7 +39,3 @@ - name: Clear the cache. ansible.builtin.include_role: name: "sync/drupal_sync_tasks/cache_clear/cache_clear-{{ project_type }}" - -- name: Restore the drush_bin variable. - ansible.builtin.set_fact: - drush_bin: "{{ _drush_bin_deploy }}" From 2eba77d9329f39124e24f8ef786150f1d04f1de9 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 24 Nov 2023 13:35:30 +0100 Subject: [PATCH 077/190] Bug fixes pr devel (#392) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../database_backup-mysql/tasks/revert-dump.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index 08059e30..6834e45c 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -1,9 +1,9 @@ --- +# unarchive module does not support archives not in tar.gz format - name: Unpack dump file. - ansible.builtin.unarchive: - src: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz" - dest: "/tmp" - remote_src: true + ansible.builtin.command: + cmd: "gunzip {{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz /tmp/{{ database.database }}-{{ previous_build_number }}.sql" + creates: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql" run_once: true when: previous_build_number > 0 From 055164c24f66f745caa83cd02c450c63e56696e9 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 24 Nov 2023 13:46:30 +0100 Subject: [PATCH 078/190] Bug fixes pr devel (#394) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../database_backup/database_backup-mysql/tasks/revert-dump.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index 6834e45c..ab729217 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -2,7 +2,7 @@ # unarchive module does not support archives not in tar.gz format - name: Unpack dump file. ansible.builtin.command: - cmd: "gunzip {{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz /tmp/{{ database.database }}-{{ previous_build_number }}.sql" + cmd: "gunzip {{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz" creates: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql" run_once: true when: previous_build_number > 0 From a8207c111e955a755779b64a0fc06088fa5bb41c Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 24 Nov 2023 16:53:21 +0100 Subject: [PATCH 079/190] Bug fixes pr devel (#395) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../database_backup-mysql/tasks/revert-dump.yml | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index ab729217..6040c73a 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -1,9 +1,16 @@ --- +- name: Move dump file to local disk. + ansible.builtin.copy: + remote_src: true + src: "{{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz" + dest: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql.gz" + run_once: true + when: previous_build_number > 0 + # unarchive module does not support archives not in tar.gz format - name: Unpack dump file. ansible.builtin.command: - cmd: "gunzip {{ mysql_backup.dumps_directory }}/{{ _mysql_host }}/{{ database.database }}-{{ previous_build_number }}.sql.gz" - creates: "/tmp/{{ database.database }}-{{ previous_build_number }}.sql" + cmd: "gunzip /tmp/{{ database.database }}-{{ previous_build_number }}.sql.gz" run_once: true when: previous_build_number > 0 From ce31e20054a0268bfe05ccab2b51d9e0beabd00e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 20 Dec 2023 18:35:58 +0100 Subject: [PATCH 080/190] Ecs deployments pr devel (#396) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * New deploy_container destroy action. * Documentation update. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- docs/roles/cache_clear/cache_clear-opcache.md | 2 +- .../database_backup/database_backup-mysql.md | 1 - docs/roles/deploy_code.md | 2 + docs/roles/deploy_container.md | 3 +- .../cache_clear/cache_clear-opcache/README.md | 2 +- .../database_backup-mysql/README.md | 1 - roles/deploy_code/README.md | 2 + roles/deploy_container/README.md | 3 +- roles/deploy_container/defaults/main.yml | 1 + .../deploy_container/tasks/action-cleanup.yml | 3 + .../deploy_container/tasks/action-create.yml | 363 +++++++++++++++++ .../deploy_container/tasks/action-destroy.yml | 254 ++++++++++++ roles/deploy_container/tasks/main.yml | 371 +----------------- 13 files changed, 640 insertions(+), 368 deletions(-) create mode 100644 roles/deploy_container/tasks/action-cleanup.yml create mode 100644 roles/deploy_container/tasks/action-create.yml create mode 100644 roles/deploy_container/tasks/action-destroy.yml diff --git a/docs/roles/cache_clear/cache_clear-opcache.md b/docs/roles/cache_clear/cache_clear-opcache.md index bb0bd05d..267485f4 100644 --- a/docs/roles/cache_clear/cache_clear-opcache.md +++ b/docs/roles/cache_clear/cache_clear-opcache.md @@ -14,7 +14,7 @@ cache_clear_opcache: # eg. # --fcgi=127.0.0.1:9000 # Leave blank to use /etc/cachetool.yml - # adapter: "127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. + # adapter: "--fcgi=127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. # Bins to clear. clear_opcache: true clear_apcu: false diff --git a/docs/roles/database_backup/database_backup-mysql.md b/docs/roles/database_backup/database_backup-mysql.md index 6637e2da..1200f97f 100644 --- a/docs/roles/database_backup/database_backup-mysql.md +++ b/docs/roles/database_backup/database_backup-mysql.md @@ -29,7 +29,6 @@ mysql_backup: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" #handling: static # optional override to the main handling method on a per database basis - ``` diff --git a/docs/roles/deploy_code.md b/docs/roles/deploy_code.md index 377ca80c..5d4cc722 100644 --- a/docs/roles/deploy_code.md +++ b/docs/roles/deploy_code.md @@ -121,6 +121,8 @@ deploy_code: service_action: reload # Trigger an API call to rebuild infra after a deploy, e.g. if you need to repack an AMI. rebuild_infra: false + # Used to skip tasks to fix ownership and permissions, drupal needs this set to true by default + fix_cleanup_perms: true # Details of API call to trigger. See api_call role. api_call: type: gitlab diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index f127bb61..752e3f59 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -41,6 +41,7 @@ It is worth noting that even if you put your containers on private subnets and c ```yaml --- deploy_container: + action: create # can also be destroy container_name: example-container container_tag: latest # tag will take format container_name:container_tag container_force_build: true # force Docker to build and tag a new image @@ -109,7 +110,7 @@ deploy_container: launch_type: FARGATE network_mode: awsvpc #volumes: [] # list of additional volumes to attach - target_group_name: example # 32 character limit + target_group_name: example # can have a maximum of 32 characters, must contain only alphanumeric characters or hyphens, and must not begin or end with a hyphen target_group_protocol: http target_group_port: 8080 # ports lower than 1024 will require the app to be configured to run as a privileged user in the Dockerfile target_group_wait_timeout: 200 # how long to wait for target group events to complete diff --git a/roles/cache_clear/cache_clear-opcache/README.md b/roles/cache_clear/cache_clear-opcache/README.md index bb0bd05d..267485f4 100644 --- a/roles/cache_clear/cache_clear-opcache/README.md +++ b/roles/cache_clear/cache_clear-opcache/README.md @@ -14,7 +14,7 @@ cache_clear_opcache: # eg. # --fcgi=127.0.0.1:9000 # Leave blank to use /etc/cachetool.yml - # adapter: "127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. + # adapter: "--fcgi=127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version. # Bins to clear. clear_opcache: true clear_apcu: false diff --git a/roles/database_backup/database_backup-mysql/README.md b/roles/database_backup/database_backup-mysql/README.md index 6637e2da..1200f97f 100644 --- a/roles/database_backup/database_backup-mysql/README.md +++ b/roles/database_backup/database_backup-mysql/README.md @@ -29,7 +29,6 @@ mysql_backup: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" #handling: static # optional override to the main handling method on a per database basis - ``` diff --git a/roles/deploy_code/README.md b/roles/deploy_code/README.md index 377ca80c..5d4cc722 100644 --- a/roles/deploy_code/README.md +++ b/roles/deploy_code/README.md @@ -121,6 +121,8 @@ deploy_code: service_action: reload # Trigger an API call to rebuild infra after a deploy, e.g. if you need to repack an AMI. rebuild_infra: false + # Used to skip tasks to fix ownership and permissions, drupal needs this set to true by default + fix_cleanup_perms: true # Details of API call to trigger. See api_call role. api_call: type: gitlab diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index f127bb61..752e3f59 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -41,6 +41,7 @@ It is worth noting that even if you put your containers on private subnets and c ```yaml --- deploy_container: + action: create # can also be destroy container_name: example-container container_tag: latest # tag will take format container_name:container_tag container_force_build: true # force Docker to build and tag a new image @@ -109,7 +110,7 @@ deploy_container: launch_type: FARGATE network_mode: awsvpc #volumes: [] # list of additional volumes to attach - target_group_name: example # 32 character limit + target_group_name: example # can have a maximum of 32 characters, must contain only alphanumeric characters or hyphens, and must not begin or end with a hyphen target_group_protocol: http target_group_port: 8080 # ports lower than 1024 will require the app to be configured to run as a privileged user in the Dockerfile target_group_wait_timeout: 200 # how long to wait for target group events to complete diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index f01dcb15..ff99d2b9 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -1,5 +1,6 @@ --- deploy_container: + action: create # can also be destroy container_name: example-container container_tag: latest # tag will take format container_name:container_tag container_force_build: true # force Docker to build and tag a new image diff --git a/roles/deploy_container/tasks/action-cleanup.yml b/roles/deploy_container/tasks/action-cleanup.yml new file mode 100644 index 00000000..57312dbe --- /dev/null +++ b/roles/deploy_container/tasks/action-cleanup.yml @@ -0,0 +1,3 @@ +--- +# @TODO we need container image cleanup here and potentially ECS Task Definition cleanup too. +# Possibly useful link: https://stackoverflow.com/a/40949364 diff --git a/roles/deploy_container/tasks/action-create.yml b/roles/deploy_container/tasks/action-create.yml new file mode 100644 index 00000000..cc06163d --- /dev/null +++ b/roles/deploy_container/tasks/action-create.yml @@ -0,0 +1,363 @@ +--- +# Build and ship a container image +- name: Create Dockerfile from template. + ansible.builtin.template: + src: "{{ deploy_container.dockerfile_template }}" + dest: "{{ deploy_container.docker_build_dir }}/Dockerfile" + delegate_to: localhost + +- name: Set Docker registry username and password. + ansible.builtin.set_fact: + _docker_registry_username: "{{ deploy_container.docker_registry_user }}" + _docker_registry_password: "{{ deploy_container.docker_registry_pass }}" + delegate_to: localhost + +- name: Fetch AWS ECR registry login token. # token valid for 12 hours + ansible.builtin.command: + cmd: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.aws_profile }}" + when: deploy_container.aws_ecr.enabled + delegate_to: localhost + register: _docker_registry_ecr_token + +- name: Set AWS ECR registry password. + ansible.builtin.set_fact: + _docker_registry_password: "{{ _docker_registry_ecr_token.stdout }}" + when: deploy_container.aws_ecr.enabled + delegate_to: localhost + +- name: Set AWS ECR registry username. + ansible.builtin.set_fact: + _docker_registry_username: "AWS" + when: deploy_container.aws_ecr.enabled + delegate_to: localhost + +- name: Remove Docker credentials file. + ansible.builtin.file: + state: absent + path: "/home/{{ deploy_user }}/.docker/config.json" + delegate_to: localhost + +- name: Log into Docker registry. + community.docker.docker_login: + registry_url: "{{ deploy_container.docker_registry_url }}" + username: "{{ _docker_registry_username }}" + password: "{{ _docker_registry_password }}" + reauthorize: true + delegate_to: localhost + +- name: Build and push container image. + community.docker.docker_image: + build: + path: "{{ deploy_container.docker_build_dir }}" + name: "{{ deploy_container.docker_registry_name }}/{{ deploy_container.container_name }}" + tag: "{{ deploy_container.container_tag | default('latest') }}" + push: true + source: build + force_source: "{{ deploy_container.container_force_build }}" + force_tag: "{{ deploy_container.container_force_build }}" + delegate_to: localhost + +# Fetch the ACM role from ce-provision +- name: Ensure the aws_acm directory exists. + ansible.builtin.file: + path: "{{ _ce_deploy_base_dir }}/roles/aws_acm/{{ item }}" + state: directory + mode: '0755' + delegate_to: localhost + with_items: + - tasks + - defaults + +- name: Fetch the aws_acm files. + ansible.builtin.get_url: + url: "https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/{{ item }}/main.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/{{ item }}/main.yml" + delegate_to: localhost + with_items: + - tasks + - defaults + +- name: Fetch the aws_acm tasks. + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/tasks/main.yml + dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/tasks/main.yml" + delegate_to: localhost + +# Gather all network information +- name: Gather VPC information. + amazon.aws.ec2_vpc_net_info: + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + region: "{{ deploy_container.aws_ecs.region }}" + filters: + "tag:Name": "{{ deploy_container.aws_ecs.vpc_name }}" + register: _aws_ecs_cluster_vpc + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + - deploy_container.aws_ecs.vpc_name is defined + - deploy_container.aws_ecs.vpc_name | length > 0 + +- name: Set the VPC id from name. + ansible.builtin.set_fact: + _aws_ecs_cluster_vpc_id: "{{ _aws_ecs_cluster_vpc.vpcs[0].vpc_id }}" + when: + - deploy_container.aws_ecs.enabled + - deploy_container.aws_ecs.vpc_name is defined + - deploy_container.aws_ecs.vpc_name | length > 0 + +- name: Use provided VPC id. + ansible.builtin.set_fact: + _aws_ecs_cluster_vpc_id: "{{ deploy_container.aws_ecs.vpc_id }}" + when: + - deploy_container.aws_ecs.enabled + - (deploy_container.aws_ecs.vpc_name is not defined or deploy_container.aws_ecs.vpc_name | length < 0) + +- name: Reset subnets lists. + ansible.builtin.set_fact: + _aws_ecs_cluster_public_subnets_ids: [] + _aws_ecs_cluster_private_subnets_ids: [] + when: deploy_container.aws_ecs.enabled + +- name: Construct list of public subnet IDs. + ansible.builtin.include_tasks: subnet-public.yml + with_items: "{{ deploy_container.aws_ecs.elb_subnets }}" + loop_control: + loop_var: subnet + when: deploy_container.aws_ecs.enabled + +- name: Construct list of private subnet IDs. + ansible.builtin.include_tasks: subnet-private.yml + with_items: "{{ deploy_container.aws_ecs.service_subnets }}" + loop_control: + loop_var: subnet + when: deploy_container.aws_ecs.enabled + +# Construct AWS supporting assets +- name: Create task definition. + community.aws.ecs_taskdefinition: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + family: "{{ deploy_container.aws_ecs.family_name }}" + execution_role_arn: "{{ deploy_container.aws_ecs.task_execution_role_arn }}" + task_role_arn: "{{ deploy_container.aws_ecs.task_role_arn | default(omit) }}" + containers: "{{ deploy_container.aws_ecs.containers }}" + launch_type: "{{ deploy_container.aws_ecs.launch_type }}" + cpu: "{{ deploy_container.aws_ecs.cpu | default(omit) }}" + memory: "{{ deploy_container.aws_ecs.memory | default(omit) }}" + state: present + network_mode: "{{ deploy_container.aws_ecs.network_mode }}" + volumes: "{{ deploy_container.aws_ecs.volumes | default(omit) }}" + force_create: "{{ deploy_container.aws_ecs.task_definition_force_create }}" + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Create a target group with IP address targets. + community.aws.elb_target_group: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit + protocol: "{{ deploy_container.aws_ecs.target_group_protocol }}" + port: "{{ deploy_container.aws_ecs.target_group_port }}" + vpc_id: "{{ _aws_ecs_cluster_vpc_id }}" + health_check_protocol: "{{ deploy_container.aws_ecs.health_check.protocol }}" + health_check_path: "{{ deploy_container.aws_ecs.health_check.path }}" + successful_response_codes: "{{ deploy_container.aws_ecs.health_check.response_codes }}" + target_type: ip + targets: "{{ deploy_container.aws_ecs.targets }}" + state: present + wait_timeout: "{{ deploy_container.aws_ecs.target_group_wait_timeout }}" + wait: true + register: _aws_ecs_target_group + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Create SSL certificate for load balancer. + ansible.builtin.include_role: + name: aws_acm + vars: + aws_acm: + region: "{{ deploy_container.aws_ecs.region }}" + aws_profile: "{{ deploy_container.aws_ecs.aws_profile }}" + tags: "{{ deploy_container.aws_ecs.tags }}" + export: false + domain_name: "{{ deploy_container.aws_ecs.domain_name }}" + extra_domains: "{{ deploy_container.aws_ecs.acm.extra_domains }}" + route_53: + aws_profile: "{{ deploy_container.aws_ecs.route_53.aws_profile }}" + zone: "{{ deploy_container.aws_ecs.route_53.zone }}" + when: + - deploy_container.aws_ecs.acm.create_cert + - deploy_container.aws_ecs.enabled + +- name: Default to provided SSL certificate ARN. + ansible.builtin.set_fact: + _ssl_certificate_ARN: "{{ deploy_container.aws_ecs.ssl_certificate_ARN }}" + when: deploy_container.aws_ecs.enabled + +- name: If provided, override SSL certificate ARN with the one received from ACM. + ansible.builtin.set_fact: + _ssl_certificate_ARN: "{{ aws_acm_certificate_arn }}" + when: + - deploy_container.aws_ecs.acm.create_cert + - deploy_container.aws_ecs.enabled + +- name: Define default ALB listeners. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners_http: + Protocol: HTTP + Port: "{{ deploy_container.aws_ecs.elb_http_port }}" + DefaultActions: + - Type: forward + TargetGroupName: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" + Rules: "{{ deploy_container.aws_ecs.elb_listener_http_rules }}" + _aws_ecs_cluster_listeners_redirect: + Protocol: HTTP + Port: "{{ deploy_container.aws_ecs.elb_http_port }}" + DefaultActions: + - Type: redirect + RedirectConfig: + Protocol: HTTPS + Host: "#{host}" + Query: "#{query}" + Path: "/#{path}" + Port: "{{ deploy_container.aws_ecs.elb_https_port }}" + StatusCode: HTTP_301 + _aws_ecs_cluster_listeners_https: + Protocol: HTTPS + Port: "{{ deploy_container.aws_ecs.elb_https_port }}" + SslPolicy: "{{ deploy_container.aws_ecs.elb_ssl_policy }}" + Certificates: + - CertificateArn: "{{ _ssl_certificate_ARN }}" + DefaultActions: + - Type: forward + TargetGroupName: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" + Rules: "{{ deploy_container.aws_ecs.elb_listener_https_rules }}" + when: deploy_container.aws_ecs.enabled + +- name: Add HTTP listeners. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners: "{{ [ _aws_ecs_cluster_listeners_http ] }}" + when: + - _ssl_certificate_ARN | length < 1 + - deploy_container.aws_ecs.enabled + +- name: Add HTTPS Listener. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners: "{{ [ _aws_ecs_cluster_listeners_redirect, _aws_ecs_cluster_listeners_https ] }}" + when: + - _ssl_certificate_ARN | length > 1 + - deploy_container.aws_ecs.enabled + +- name: Add custom Listeners. + ansible.builtin.set_fact: + _aws_ecs_cluster_listeners: "{{ _aws_ecs_cluster_listeners + deploy_container.aws_ecs.elb_listeners }}" + when: + - deploy_container.aws_ecs.elb_listeners is defined + - deploy_container.aws_ecs.elb_listeners | length + - deploy_container.aws_ecs.enabled + +- name: Create an ALB. + amazon.aws.elb_application_lb: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit + state: present + tags: "{{ deploy_container.aws_ecs.tags }}" + subnets: "{{ _aws_ecs_cluster_public_subnets_ids }}" + security_groups: "{{ deploy_container.aws_ecs.elb_security_groups }}" + listeners: "{{ _aws_ecs_cluster_listeners }}" + idle_timeout: "{{ deploy_container.aws_ecs.elb_idle_timeout }}" + ip_address_type: "{{ deploy_container.aws_ecs.elb_ip_address_type }}" + register: _aws_ecs_cluster_alb + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Set task definition name. + ansible.builtin.set_fact: + _aws_ecs_service_task_definition: "{{ deploy_container.aws_ecs.family_name }}" + when: deploy_container.aws_ecs.enabled + +- name: Set task definition revision if applicable. + ansible.builtin.set_fact: + _aws_ecs_service_task_definition: "{{ deploy_container.aws_ecs.family_name }}:{{ deploy_container.aws_ecs.task_definition_revision }}" + when: + - deploy_container.aws_ecs.task_definition_revision | length > 0 + - deploy_container.aws_ecs.enabled + +- name: Create ECS service. + community.aws.ecs_service: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + state: present + name: "{{ deploy_container.aws_ecs.family_name }}" + cluster: "{{ deploy_container.aws_ecs.cluster_name }}" + task_definition: "{{ _aws_ecs_service_task_definition }}" + desired_count: "{{ deploy_container.aws_ecs.task_count }}" + launch_type: "{{ deploy_container.aws_ecs.launch_type }}" + platform_version: LATEST + load_balancers: # see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LoadBalancer.html + - containerName: "{{ deploy_container.container_name }}" + containerPort: "{{ deploy_container.aws_ecs.target_group_port }}" + targetGroupArn: "{{ _aws_ecs_target_group.target_group_arn }}" + network_configuration: + subnets: "{{ _aws_ecs_cluster_private_subnets_ids }}" # internal private subnet + security_groups: "{{ deploy_container.aws_ecs.security_groups }}" + assign_public_ip: "{{ deploy_container.aws_ecs.service_public_container_ip }}" + tags: "{{ deploy_container.aws_ecs.tags }}" + enable_execute_command: "{{ deploy_container.aws_ecs.service_enable_ssm }}" + force_new_deployment: "{{ deploy_container.aws_ecs.service_force_refresh }}" + wait: true + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Create target tracking scaling policy for ECS service. + community.aws.application_autoscaling_policy: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + state: present + policy_name: "{{ deploy_container.aws_ecs.family_name }}" + service_namespace: ecs + resource_id: "service/{{ deploy_container.aws_ecs.cluster_name }}/{{ deploy_container.aws_ecs.family_name }}" + scalable_dimension: ecs:service:DesiredCount + minimum_tasks: "{{ deploy_container.aws_ecs.task_minimum_count }}" + maximum_tasks: "{{ deploy_container.aws_ecs.task_maximum_count }}" + policy_type: TargetTrackingScaling + target_tracking_scaling_policy_configuration: + PredefinedMetricSpecification: + PredefinedMetricType: "{{ deploy_container.aws_ecs.service_autoscale_metric_type }}" + ScaleInCooldown: "{{ deploy_container.aws_ecs.service_autoscale_up_cooldown }}" + ScaleOutCooldown: "{{ deploy_container.aws_ecs.service_autoscale_down_cooldown }}" + DisableScaleIn: false + TargetValue: "{{ deploy_container.aws_ecs.service_autoscale_target_value }}" + delegate_to: localhost + when: deploy_container.aws_ecs.enabled + +- name: Initialise the domains loop var with main domain entry DNS settings. + ansible.builtin.set_fact: + _aws_ecs_cluster_dns_all_domains: + - domain: "{{ deploy_container.aws_ecs.domain_name }}" + zone: "{{ deploy_container.aws_ecs.route_53.zone }}" + aws_profile: "{{ deploy_container.aws_ecs.route_53.aws_profile }}" + when: deploy_container.aws_ecs.enabled + +- name: Add extra_domains so we can loop through DNS records. + ansible.builtin.set_fact: + _aws_ecs_cluster_dns_all_domains: "{{ _aws_ecs_cluster_dns_all_domains + [{'domain': item.domain, 'zone': item.zone, 'aws_profile': item.aws_profile}] }}" + loop: "{{ deploy_container.aws_ecs.acm.extra_domains }}" + when: + - deploy_container.aws_ecs.acm.extra_domains | length > 0 + - deploy_container.aws_ecs.enabled + +- name: Add DNS records in Route 53. + amazon.aws.route53: + state: present + profile: "{{ item.aws_profile }}" + zone: "{{ item.zone }}" + record: "{{ item.domain }}" + type: CNAME + value: "{{ _aws_ecs_cluster_alb.dns_name }}" + overwrite: true + loop: "{{ _aws_ecs_cluster_dns_all_domains }}" + when: + - deploy_container.aws_ecs.route_53.zone | length > 0 + - deploy_container.aws_ecs.enabled diff --git a/roles/deploy_container/tasks/action-destroy.yml b/roles/deploy_container/tasks/action-destroy.yml new file mode 100644 index 00000000..b0b6bd99 --- /dev/null +++ b/roles/deploy_container/tasks/action-destroy.yml @@ -0,0 +1,254 @@ +--- +# Be sure to include your deploy_container variables in your playbook +- name: Set Docker registry username and password. + ansible.builtin.set_fact: + _docker_registry_username: "{{ deploy_container.docker_registry_user }}" + _docker_registry_password: "{{ deploy_container.docker_registry_pass }}" + delegate_to: localhost + +- name: Fetch AWS ECR registry login token. # token valid for 12 hours + ansible.builtin.command: + cmd: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.aws_profile }}" + when: deploy_container.aws_ecr.enabled + delegate_to: localhost + register: _docker_registry_ecr_token + +- name: Set AWS ECR registry password. + ansible.builtin.set_fact: + _docker_registry_password: "{{ _docker_registry_ecr_token.stdout }}" + when: deploy_container.aws_ecr.enabled + delegate_to: localhost + +- name: Set AWS ECR registry username. + ansible.builtin.set_fact: + _docker_registry_username: "AWS" + when: deploy_container.aws_ecr.enabled + delegate_to: localhost + +- name: Remove Docker credentials file. + ansible.builtin.file: + state: absent + path: "/home/{{ deploy_user }}/.docker/config.json" + delegate_to: localhost + +- name: Log into Docker registry. + community.docker.docker_login: + registry_url: "{{ deploy_container.docker_registry_url }}" + username: "{{ _docker_registry_username }}" + password: "{{ _docker_registry_password }}" + reauthorize: true + delegate_to: localhost + +- name: Destroy matching container images. + community.docker.docker_image: + name: "{{ deploy_container.docker_registry_name }}/{{ deploy_container.container_name }}" + tag: "{{ deploy_container.container_tag | default('latest') }}" + force_absent: true + state: absent + delegate_to: localhost + +# Destroy AWS services +- name: Get minimal ALB information before we destroy it. + amazon.aws.elb_application_lb_info: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + names: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit + #include_attributes: false # @TODO - these attributes added in amazon.aws 7.0.0 + #include_listeners: false + #include_listener_rules: false + register: _aws_ecs_cluster_alb + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + +- name: Destroy ALB. + amazon.aws.elb_application_lb: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit + state: absent + wait: true + when: + - deploy_container.aws_ecs.enabled + +- name: Destroy target group. + community.aws.elb_target_group: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit + state: absent + wait: true + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + +- name: Reduce task count to zero on ECS service. + community.aws.ecs_service: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + state: present + name: "{{ deploy_container.aws_ecs.family_name }}" + cluster: "{{ deploy_container.aws_ecs.cluster_name }}" + desired_count: 0 + force_new_deployment: true + wait: true + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + +- name: Destroy ECS service. + community.aws.ecs_service: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + state: absent + name: "{{ deploy_container.aws_ecs.family_name }}" + cluster: "{{ deploy_container.aws_ecs.cluster_name }}" + wait: true + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + +# @TODO: We cannot currently use the module for this +# See feature request: https://github.com/ansible-collections/community.aws/issues/2023 +#- name: Get task definition details. +# community.aws.ecs_taskdefinition_info: +# region: "{{ deploy_container.aws_ecs.region }}" +# profile: "{{ deploy_container.aws_ecs.aws_profile }}" +# task_definition: "{{ deploy_container.aws_ecs.family_name }}" +# register: _task_definition_info +# delegate_to: localhost + +- name: Ensure the task definitions ARN list variable is empty. + ansible.builtin.set_fact: + _task_definition_arns_list: [] + when: + - deploy_container.aws_ecs.enabled + +- name: Get active task definition details. + ansible.builtin.command: + cmd: "aws ecs list-task-definitions --status ACTIVE --family-prefix {{ deploy_container.aws_ecs.family_name }} --region {{ deploy_container.aws_ecs.region }} --profile {{ deploy_container.aws_ecs.aws_profile }}" + register: _task_definition_arns_raw + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + +- name: Convert CLI output of active ARNs to a YAML variable. + ansible.builtin.set_fact: + _task_definition_arns: "{{ _task_definition_arns_raw.stdout | from_json }}" + when: + - deploy_container.aws_ecs.enabled + +- name: Create a clean list of task definition ARNs. + ansible.builtin.set_fact: + _task_definition_arns_list: "{{ _task_definition_arns.taskDefinitionArns }}" + when: + - deploy_container.aws_ecs.enabled + +- name: Get inactive task definition details. + ansible.builtin.command: + cmd: "aws ecs list-task-definitions --status INACTIVE --family-prefix {{ deploy_container.aws_ecs.family_name }} --region {{ deploy_container.aws_ecs.region }} --profile {{ deploy_container.aws_ecs.aws_profile }}" + register: _task_definition_arns_raw + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + +- name: Convert CLI output of inactive ARNs to a YAML variable. + ansible.builtin.set_fact: + _task_definition_arns: "{{ _task_definition_arns_raw.stdout | from_json }}" + when: + - deploy_container.aws_ecs.enabled + +- name: Add inactive task definition ARNs to the YAML list. + ansible.builtin.set_fact: + _task_definition_arns_list: "{{ _task_definition_arns_list + _task_definition_arns.taskDefinitionArns }}" + when: + - deploy_container.aws_ecs.enabled + +- name: Deregister task definitions. + community.aws.ecs_taskdefinition: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + arn: "{{ item }}" + containers: "{{ deploy_container.aws_ecs.containers }}" + state: absent + delegate_to: localhost + with_items: "{{ _task_definition_arns_list }}" + when: + - deploy_container.aws_ecs.enabled + +- name: Delete task definitions. + ansible.builtin.command: + cmd: "aws ecs delete-task-definitions --task-definitions {{ item }} --region {{ deploy_container.aws_ecs.region }} --profile {{ deploy_container.aws_ecs.aws_profile }}" + delegate_to: localhost + with_items: "{{ _task_definition_arns_list }}" + when: + - deploy_container.aws_ecs.enabled + +- name: Destroy scaling policy for ECS service. + community.aws.application_autoscaling_policy: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + state: absent + policy_name: "{{ deploy_container.aws_ecs.family_name }}" + service_namespace: ecs + policy_type: TargetTrackingScaling + resource_id: "service/{{ deploy_container.aws_ecs.cluster_name }}/{{ deploy_container.aws_ecs.family_name }}" + scalable_dimension: ecs:service:DesiredCount + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + +# Clean up SSL certificates +- name: Delete the main ACM certificate. + community.aws.acm_certificate: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + domain_name: "{{ deploy_container.aws_ecs.domain_name }}" + state: absent + delegate_to: localhost + when: + - deploy_container.aws_ecs.enabled + - deploy_container.aws_ecs.acm.create_cert + +- name: Delete any extra ACM certificates. + community.aws.acm_certificate: + region: "{{ deploy_container.aws_ecs.region }}" + profile: "{{ deploy_container.aws_ecs.aws_profile }}" + domain_name: "{{ item.domain }}" + state: absent + with_items: "{{ deploy_container.aws_ecs.acm.extra_domains }}" + when: + - deploy_container.aws_ecs.acm.extra_domains | length > 0 + - deploy_container.aws_ecs.enabled + delegate_to: localhost + +# Clean up DNS +- name: Initialise the domains loop var with main domain entry DNS settings. + ansible.builtin.set_fact: + _aws_ecs_cluster_dns_all_domains: + - domain: "{{ deploy_container.aws_ecs.domain_name }}" + zone: "{{ deploy_container.aws_ecs.route_53.zone }}" + aws_profile: "{{ deploy_container.aws_ecs.route_53.aws_profile }}" + when: + - deploy_container.aws_ecs.enabled + +- name: Add extra_domains so we can loop through DNS records. + ansible.builtin.set_fact: + _aws_ecs_cluster_dns_all_domains: "{{ _aws_ecs_cluster_dns_all_domains + [{'domain': item.domain, 'zone': item.zone, 'aws_profile': item.aws_profile}] }}" + loop: "{{ deploy_container.aws_ecs.acm.extra_domains }}" + when: + - deploy_container.aws_ecs.enabled + - deploy_container.aws_ecs.acm.extra_domains | length > 0 + +- name: Remove DNS records in Route 53. + amazon.aws.route53: + state: absent + profile: "{{ item.aws_profile }}" + zone: "{{ item.zone }}" + record: "{{ item.domain }}" + type: CNAME + value: "{{ _aws_ecs_cluster_alb.load_balancers[0].dns_name }}" + loop: "{{ _aws_ecs_cluster_dns_all_domains }}" + when: + - deploy_container.aws_ecs.enabled + - deploy_container.aws_ecs.route_53.zone | length > 0 diff --git a/roles/deploy_container/tasks/main.yml b/roles/deploy_container/tasks/main.yml index 6c68bebd..ce3e618f 100644 --- a/roles/deploy_container/tasks/main.yml +++ b/roles/deploy_container/tasks/main.yml @@ -1,363 +1,10 @@ --- -# Build and ship a container image -- name: Create Dockerfile from template. - ansible.builtin.template: - src: "{{ deploy_container.dockerfile_template }}" - dest: "{{ deploy_container.docker_build_dir }}/Dockerfile" - delegate_to: localhost - -- name: Set Docker registry username and password. - ansible.builtin.set_fact: - _docker_registry_username: "{{ deploy_container.docker_registry_user }}" - _docker_registry_password: "{{ deploy_container.docker_registry_pass }}" - delegate_to: localhost - -- name: Fetch AWS ECR registry login token. # token valid for 12 hours - ansible.builtin.command: - cmd: "aws ecr get-login-password --region {{ deploy_container.aws_ecr.region }} --profile {{ deploy_container.aws_ecr.aws_profile }}" - when: deploy_container.aws_ecr.enabled - delegate_to: localhost - register: _docker_registry_ecr_token - -- name: Set AWS ECR registry password. - ansible.builtin.set_fact: - _docker_registry_password: "{{ _docker_registry_ecr_token.stdout }}" - when: deploy_container.aws_ecr.enabled - delegate_to: localhost - -- name: Set AWS ECR registry username. - ansible.builtin.set_fact: - _docker_registry_username: "AWS" - when: deploy_container.aws_ecr.enabled - delegate_to: localhost - -- name: Remove Docker credentials file. - ansible.builtin.file: - state: absent - path: "/home/{{ deploy_user }}/.docker/config.json" - delegate_to: localhost - -- name: Log into Docker registry. - community.docker.docker_login: - registry_url: "{{ deploy_container.docker_registry_url }}" - username: "{{ _docker_registry_username }}" - password: "{{ _docker_registry_password }}" - reauthorize: true - delegate_to: localhost - -- name: Build and push container image. - community.docker.docker_image: - build: - path: "{{ deploy_container.docker_build_dir }}" - name: "{{ deploy_container.docker_registry_name }}/{{ deploy_container.container_name }}" - tag: "{{ deploy_container.container_tag | default('latest') }}" - push: true - source: build - force_source: "{{ deploy_container.container_force_build }}" - force_tag: "{{ deploy_container.container_force_build }}" - delegate_to: localhost - -# Fetch the ACM role from ce-provision -- name: Ensure the aws_acm directory exists. - ansible.builtin.file: - path: "{{ _ce_deploy_base_dir }}/roles/aws_acm/{{ item }}" - state: directory - mode: '0755' - delegate_to: localhost - with_items: - - tasks - - defaults - -- name: Fetch the aws_acm files. - ansible.builtin.get_url: - url: "https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/{{ item }}/main.yml" - dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/{{ item }}/main.yml" - delegate_to: localhost - with_items: - - tasks - - defaults - -- name: Fetch the aws_acm tasks. - ansible.builtin.get_url: - url: https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/tasks/main.yml - dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/tasks/main.yml" - delegate_to: localhost - -# Gather all network information -- name: Gather VPC information. - amazon.aws.ec2_vpc_net_info: - profile: "{{ deploy_container.aws_ecs.aws_profile }}" - region: "{{ deploy_container.aws_ecs.region }}" - filters: - "tag:Name": "{{ deploy_container.aws_ecs.vpc_name }}" - register: _aws_ecs_cluster_vpc - delegate_to: localhost - when: - - deploy_container.aws_ecs.enabled - - deploy_container.aws_ecs.vpc_name is defined - - deploy_container.aws_ecs.vpc_name | length > 0 - -- name: Set the VPC id from name. - ansible.builtin.set_fact: - _aws_ecs_cluster_vpc_id: "{{ _aws_ecs_cluster_vpc.vpcs[0].vpc_id }}" - when: - - deploy_container.aws_ecs.enabled - - deploy_container.aws_ecs.vpc_name is defined - - deploy_container.aws_ecs.vpc_name | length > 0 - -- name: Use provided VPC id. - ansible.builtin.set_fact: - _aws_ecs_cluster_vpc_id: "{{ deploy_container.aws_ecs.vpc_id }}" - when: - - deploy_container.aws_ecs.enabled - - (deploy_container.aws_ecs.vpc_name is not defined or deploy_container.aws_ecs.vpc_name | length < 0) - -- name: Reset subnets lists. - ansible.builtin.set_fact: - _aws_ecs_cluster_public_subnets_ids: [] - _aws_ecs_cluster_private_subnets_ids: [] - when: deploy_container.aws_ecs.enabled - -- name: Construct list of public subnet IDs. - ansible.builtin.include_tasks: subnet-public.yml - with_items: "{{ deploy_container.aws_ecs.elb_subnets }}" - loop_control: - loop_var: subnet - when: deploy_container.aws_ecs.enabled - -- name: Construct list of private subnet IDs. - ansible.builtin.include_tasks: subnet-private.yml - with_items: "{{ deploy_container.aws_ecs.service_subnets }}" - loop_control: - loop_var: subnet - when: deploy_container.aws_ecs.enabled - -# Construct AWS supporting assets -- name: Create task definition. - community.aws.ecs_taskdefinition: - region: "{{ deploy_container.aws_ecs.region }}" - profile: "{{ deploy_container.aws_ecs.aws_profile }}" - family: "{{ deploy_container.aws_ecs.family_name }}" - execution_role_arn: "{{ deploy_container.aws_ecs.task_execution_role_arn }}" - task_role_arn: "{{ deploy_container.aws_ecs.task_role_arn | default(omit) }}" - containers: "{{ deploy_container.aws_ecs.containers }}" - launch_type: "{{ deploy_container.aws_ecs.launch_type }}" - cpu: "{{ deploy_container.aws_ecs.cpu | default(omit) }}" - memory: "{{ deploy_container.aws_ecs.memory | default(omit) }}" - state: present - network_mode: "{{ deploy_container.aws_ecs.network_mode }}" - volumes: "{{ deploy_container.aws_ecs.volumes | default(omit) }}" - force_create: "{{ deploy_container.aws_ecs.task_definition_force_create }}" - delegate_to: localhost - when: deploy_container.aws_ecs.enabled - -- name: Create a target group with IP address targets. - community.aws.elb_target_group: - region: "{{ deploy_container.aws_ecs.region }}" - profile: "{{ deploy_container.aws_ecs.aws_profile }}" - name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit - protocol: "{{ deploy_container.aws_ecs.target_group_protocol }}" - port: "{{ deploy_container.aws_ecs.target_group_port }}" - vpc_id: "{{ _aws_ecs_cluster_vpc_id }}" - health_check_protocol: "{{ deploy_container.aws_ecs.health_check.protocol }}" - health_check_path: "{{ deploy_container.aws_ecs.health_check.path }}" - successful_response_codes: "{{ deploy_container.aws_ecs.health_check.response_codes }}" - target_type: ip - targets: "{{ deploy_container.aws_ecs.targets }}" - state: present - wait_timeout: "{{ deploy_container.aws_ecs.target_group_wait_timeout }}" - wait: true - register: _aws_ecs_target_group - delegate_to: localhost - when: deploy_container.aws_ecs.enabled - -- name: Create SSL certificate for load balancer. - ansible.builtin.include_role: - name: aws_acm - vars: - aws_acm: - region: "{{ deploy_container.aws_ecs.region }}" - aws_profile: "{{ deploy_container.aws_ecs.aws_profile }}" - tags: "{{ deploy_container.aws_ecs.tags }}" - export: false - domain_name: "{{ deploy_container.aws_ecs.domain_name }}" - extra_domains: "{{ deploy_container.aws_ecs.acm.extra_domains }}" - route_53: - aws_profile: "{{ deploy_container.aws_ecs.route_53.aws_profile }}" - zone: "{{ deploy_container.aws_ecs.route_53.zone }}" - when: - - deploy_container.aws_ecs.acm.create_cert - - deploy_container.aws_ecs.enabled - -- name: Default to provided SSL certificate ARN. - ansible.builtin.set_fact: - _ssl_certificate_ARN: "{{ deploy_container.aws_ecs.ssl_certificate_ARN }}" - when: deploy_container.aws_ecs.enabled - -- name: If provided, override SSL certificate ARN with the one received from ACM. - ansible.builtin.set_fact: - _ssl_certificate_ARN: "{{ aws_acm_certificate_arn }}" - when: - - deploy_container.aws_ecs.acm.create_cert - - deploy_container.aws_ecs.enabled - -- name: Define default ALB listeners. - ansible.builtin.set_fact: - _aws_ecs_cluster_listeners_http: - Protocol: HTTP - Port: "{{ deploy_container.aws_ecs.elb_http_port }}" - DefaultActions: - - Type: forward - TargetGroupName: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" - Rules: "{{ deploy_container.aws_ecs.elb_listener_http_rules }}" - _aws_ecs_cluster_listeners_redirect: - Protocol: HTTP - Port: "{{ deploy_container.aws_ecs.elb_http_port }}" - DefaultActions: - - Type: redirect - RedirectConfig: - Protocol: HTTPS - Host: "#{host}" - Query: "#{query}" - Path: "/#{path}" - Port: "{{ deploy_container.aws_ecs.elb_https_port }}" - StatusCode: HTTP_301 - _aws_ecs_cluster_listeners_https: - Protocol: HTTPS - Port: "{{ deploy_container.aws_ecs.elb_https_port }}" - SslPolicy: "{{ deploy_container.aws_ecs.elb_ssl_policy }}" - Certificates: - - CertificateArn: "{{ _ssl_certificate_ARN }}" - DefaultActions: - - Type: forward - TargetGroupName: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" - Rules: "{{ deploy_container.aws_ecs.elb_listener_https_rules }}" - when: deploy_container.aws_ecs.enabled - -- name: Add HTTP listeners. - ansible.builtin.set_fact: - _aws_ecs_cluster_listeners: "{{ [ _aws_ecs_cluster_listeners_http ] }}" - when: - - _ssl_certificate_ARN | length < 1 - - deploy_container.aws_ecs.enabled - -- name: Add HTTPS Listener. - ansible.builtin.set_fact: - _aws_ecs_cluster_listeners: "{{ [ _aws_ecs_cluster_listeners_redirect, _aws_ecs_cluster_listeners_https ] }}" - when: - - _ssl_certificate_ARN | length > 1 - - deploy_container.aws_ecs.enabled - -- name: Add custom Listeners. - ansible.builtin.set_fact: - _aws_ecs_cluster_listeners: "{{ _aws_ecs_cluster_listeners + deploy_container.aws_ecs.elb_listeners }}" - when: - - deploy_container.aws_ecs.elb_listeners is defined - - deploy_container.aws_ecs.elb_listeners | length - - deploy_container.aws_ecs.enabled - -- name: Create an ALB. - amazon.aws.elb_application_lb: - region: "{{ deploy_container.aws_ecs.region }}" - profile: "{{ deploy_container.aws_ecs.aws_profile }}" - name: "{{ deploy_container.aws_ecs.target_group_name | truncate(32, true, '', 0) }}" # 32 char limit - state: present - tags: "{{ deploy_container.aws_ecs.tags }}" - subnets: "{{ _aws_ecs_cluster_public_subnets_ids }}" - security_groups: "{{ deploy_container.aws_ecs.elb_security_groups }}" - listeners: "{{ _aws_ecs_cluster_listeners }}" - idle_timeout: "{{ deploy_container.aws_ecs.elb_idle_timeout }}" - ip_address_type: "{{ deploy_container.aws_ecs.elb_ip_address_type }}" - register: _aws_ecs_cluster_alb - delegate_to: localhost - when: deploy_container.aws_ecs.enabled - -- name: Set task definition name. - ansible.builtin.set_fact: - _aws_ecs_service_task_definition: "{{ deploy_container.aws_ecs.family_name }}" - when: deploy_container.aws_ecs.enabled - -- name: Set task definition revision if applicable. - ansible.builtin.set_fact: - _aws_ecs_service_task_definition: "{{ deploy_container.aws_ecs.family_name }}:{{ deploy_container.aws_ecs.task_definition_revision }}" - when: - - deploy_container.aws_ecs.task_definition_revision | length > 0 - - deploy_container.aws_ecs.enabled - -- name: Create ECS service. - community.aws.ecs_service: - region: "{{ deploy_container.aws_ecs.region }}" - profile: "{{ deploy_container.aws_ecs.aws_profile }}" - state: present - name: "{{ deploy_container.aws_ecs.family_name }}" - cluster: "{{ deploy_container.aws_ecs.cluster_name }}" - task_definition: "{{ _aws_ecs_service_task_definition }}" - desired_count: "{{ deploy_container.aws_ecs.task_count }}" - launch_type: "{{ deploy_container.aws_ecs.launch_type }}" - platform_version: LATEST - load_balancers: # see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LoadBalancer.html - - containerName: "{{ deploy_container.container_name }}" - containerPort: "{{ deploy_container.aws_ecs.target_group_port }}" - targetGroupArn: "{{ _aws_ecs_target_group.target_group_arn }}" - network_configuration: - subnets: "{{ _aws_ecs_cluster_private_subnets_ids }}" # internal private subnet - security_groups: "{{ deploy_container.aws_ecs.security_groups }}" - assign_public_ip: "{{ deploy_container.aws_ecs.service_public_container_ip }}" - tags: "{{ deploy_container.aws_ecs.tags }}" - enable_execute_command: "{{ deploy_container.aws_ecs.service_enable_ssm }}" - force_new_deployment: "{{ deploy_container.aws_ecs.service_force_refresh }}" - wait: true - delegate_to: localhost - when: deploy_container.aws_ecs.enabled - -- name: Create target tracking scaling policy for ECS service. - community.aws.application_autoscaling_policy: - region: "{{ deploy_container.aws_ecs.region }}" - profile: "{{ deploy_container.aws_ecs.aws_profile }}" - state: present - policy_name: "{{ deploy_container.aws_ecs.family_name }}" - service_namespace: ecs - resource_id: "service/{{ deploy_container.aws_ecs.cluster_name }}/{{ deploy_container.aws_ecs.family_name }}" - scalable_dimension: ecs:service:DesiredCount - minimum_tasks: "{{ deploy_container.aws_ecs.task_minimum_count }}" - maximum_tasks: "{{ deploy_container.aws_ecs.task_maximum_count }}" - policy_type: TargetTrackingScaling - target_tracking_scaling_policy_configuration: - PredefinedMetricSpecification: - PredefinedMetricType: "{{ deploy_container.aws_ecs.service_autoscale_metric_type }}" - ScaleInCooldown: "{{ deploy_container.aws_ecs.service_autoscale_up_cooldown }}" - ScaleOutCooldown: "{{ deploy_container.aws_ecs.service_autoscale_down_cooldown }}" - DisableScaleIn: false - TargetValue: "{{ deploy_container.aws_ecs.service_autoscale_target_value }}" - delegate_to: localhost - when: deploy_container.aws_ecs.enabled - -- name: Initialise the domains loop var with main domain entry DNS settings. - ansible.builtin.set_fact: - _aws_ecs_cluster_dns_all_domains: - - domain: "{{ deploy_container.aws_ecs.domain_name }}" - zone: "{{ deploy_container.aws_ecs.route_53.zone }}" - aws_profile: "{{ deploy_container.aws_ecs.route_53.aws_profile }}" - when: deploy_container.aws_ecs.enabled - -- name: Add extra_domains so we can loop through DNS records. - ansible.builtin.set_fact: - _aws_ecs_cluster_dns_all_domains: "{{ _aws_ecs_cluster_dns_all_domains + [{'domain': item.domain, 'zone': item.zone, 'aws_profile': item.aws_profile}] }}" - loop: "{{ deploy_container.aws_ecs.acm.extra_domains }}" - when: - - deploy_container.aws_ecs.acm.extra_domains | length > 0 - - deploy_container.aws_ecs.enabled - -- name: Add DNS records in Route 53. - amazon.aws.route53: - state: present - profile: "{{ item.aws_profile }}" - zone: "{{ item.zone }}" - record: "{{ item.domain }}" - type: CNAME - value: "{{ _aws_ecs_cluster_alb.dns_name }}" - overwrite: true - loop: "{{ _aws_ecs_cluster_dns_all_domains }}" - when: - - deploy_container.aws_ecs.route_53.zone | length > 0 - - deploy_container.aws_ecs.enabled \ No newline at end of file +- name: Execute a container deployment. + ansible.builtin.include_tasks: + file: "action-{{ deploy_container.action }}.yml" + when: deploy_operation == 'deploy' + +- name: Cleanup dangling containers. + ansible.builtin.include_tasks: + file: action-cleanup.yml + when: deploy_operation == 'cleanup' From b74ce97d0c78c0d773d2359411017719fc245f28 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 20 Dec 2023 18:49:26 +0100 Subject: [PATCH 081/190] Ecs deployments pr devel (#398) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- docs/roles/deploy_container.md | 2 ++ roles/deploy_container/README.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 752e3f59..06847722 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -8,6 +8,8 @@ sudo usermod -aG docker deploy This can be handled automatically by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. +If you set the `deploy_container.action` to `destroy` then the role will also take care of tidying up containers. If it is an AWS ECS deployment then it will also tidy up the ECS service for you. + ## AWS IAM requirements AWS integration requires the AWS CLI user provided for `ce-deploy` to have certain managed AWS policies attached. diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 752e3f59..06847722 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -8,6 +8,8 @@ sudo usermod -aG docker deploy This can be handled automatically by [`ce-provision`](https://github.com/codeenigma/ce-provision) using the `ce_deploy` and `docker_ce` roles. +If you set the `deploy_container.action` to `destroy` then the role will also take care of tidying up containers. If it is an AWS ECS deployment then it will also tidy up the ECS service for you. + ## AWS IAM requirements AWS integration requires the AWS CLI user provided for `ce-deploy` to have certain managed AWS policies attached. From 6da34b3f07463aba33fb51e1bf49c22a2bb7d2a7 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 11 Jan 2024 10:56:15 +0100 Subject: [PATCH 082/190] Bug fixes pr devel (#399) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- scripts/_common.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/scripts/_common.sh b/scripts/_common.sh index 04e14c18..a34ff141 100755 --- a/scripts/_common.sh +++ b/scripts/_common.sh @@ -36,6 +36,13 @@ BUILD_TRACK_DIR="$OWN_DIR/track" if [ ! -d "$BUILD_TRACK_DIR" ]; then mkdir "$BUILD_TRACK_DIR" fi +# Load the contents of profile.d in case we added items to $PATH there. +if [ -n "$(ls -A /etc/profile.d)" ]; then + for f in /etc/profile.d/*; do + # shellcheck source=/dev/null + . "$f" + done +fi ANSIBLE_LOCATION=$(command -v ansible) # Load the contents of profile.d in case we added items to $PATH there. for f in /etc/profile.d/*; do From 6ba4fa3d49f9c3eccf773361e111f39b9bd75130 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 18 Jan 2024 09:19:53 +0100 Subject: [PATCH 083/190] Bug fixes pr devel (#401) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- scripts/_common.sh | 21 ++++++++++++++++++++- scripts/build.sh | 2 ++ scripts/cleanup.sh | 2 ++ scripts/deploy.sh | 2 ++ scripts/revert.sh | 2 ++ 5 files changed, 28 insertions(+), 1 deletion(-) diff --git a/scripts/_common.sh b/scripts/_common.sh index a34ff141..07aa7368 100755 --- a/scripts/_common.sh +++ b/scripts/_common.sh @@ -14,6 +14,8 @@ PREVIOUS_BUILD_NUMBER="" CURRENT_BUILD_NUMBER="" ANSIBLE_EXTRA_VARS="" ANSIBLE_DEFAULT_EXTRA_VARS="" +ANSIBLE_PATH="" +PYTHON_INTERPRETER="" BUILD_WORKSPACE="" BUILD_TRACK_FILE="" BUILD_ID="" @@ -103,6 +105,14 @@ parse_options(){ shift BOTO_PROFILE="$1" ;; + "--ansible-path") + shift + ANSIBLE_PATH="$1" + ;; + "--python-interpreter") + shift + PYTHON_INTERPRETER="$1" + ;; "--build-id") shift BUILD_ID="$1" @@ -128,7 +138,11 @@ get_build_workspace(){ # Common extra-vars to pass to Ansible. get_ansible_defaults_vars(){ - ANSIBLE_DEFAULT_EXTRA_VARS="{_ce_deploy_base_dir: $OWN_DIR, _ce_deploy_build_dir: $BUILD_WORKSPACE, _ce_deploy_build_tmp_dir: $BUILD_TMP_DIR, _ce_deploy_data_dir: $ANSIBLE_DATA_DIR, _ce_deploy_ansible_location: $ANSIBLE_LOCATION, build_number: $CURRENT_BUILD_NUMBER, previous_known_build_number: $PREVIOUS_BUILD_NUMBER}" + if [ -n "$PYTHON_INTERPRETER" ]; then + ANSIBLE_DEFAULT_EXTRA_VARS="{ansible_python_interpreter: $PYTHON_INTERPRETER, _ce_deploy_base_dir: $OWN_DIR, _ce_deploy_build_dir: $BUILD_WORKSPACE, _ce_deploy_build_tmp_dir: $BUILD_TMP_DIR, _ce_deploy_data_dir: $ANSIBLE_DATA_DIR, _ce_deploy_ansible_location: $ANSIBLE_LOCATION, build_number: $CURRENT_BUILD_NUMBER, previous_known_build_number: $PREVIOUS_BUILD_NUMBER}" + else + ANSIBLE_DEFAULT_EXTRA_VARS="{_ce_deploy_base_dir: $OWN_DIR, _ce_deploy_build_dir: $BUILD_WORKSPACE, _ce_deploy_build_tmp_dir: $BUILD_TMP_DIR, _ce_deploy_data_dir: $ANSIBLE_DATA_DIR, _ce_deploy_ansible_location: $ANSIBLE_LOCATION, build_number: $CURRENT_BUILD_NUMBER, previous_known_build_number: $PREVIOUS_BUILD_NUMBER}" + fi } # Fetch previous build number from track file. @@ -169,6 +183,11 @@ cleanup_build_tmp_dir(){ # Call Ansible playbook to ensure host exists. ansible_host_check(){ if [ -n "$TARGET_DEPLOY_HOST" ]; then + if [ -z "$ANSIBLE_PATH" ]; then + ANSIBLE_BIN=$(command -v ansible-playbook) + else + ANSIBLE_BIN="$ANSIBLE_PATH/ansible-playbook" + fi ANSIBLE_BIN=$(command -v ansible-playbook) ANSIBLE_CMD="$ANSIBLE_BIN $OWN_DIR/scripts/host-check.yml" if [ "$VERBOSE" = "yes" ]; then diff --git a/scripts/build.sh b/scripts/build.sh index 93f36887..a29ec43b 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -21,6 +21,8 @@ usage(){ echo 'Available options:' echo '--host: Valid Ansible hostname, if you want to run a host check. Can also be a group name.' echo '--ansible-extra-vars: Variable to pass as --extra-vars arguments to ansible-playbook. Make sure to escape them properly.' + echo '--ansible-path: Pass the path to the directory containing the Ansible binaries if you are not using the version of Ansible in PATH.' + echo '--python-interpreter: When using Python virtual environments Ansible may not correctly determine the Python interpreter, use this to set it manually.' echo '--previous-stable-build-number: an incremental build number that ' echo '--dry-run: Do not perform any action but run the playbooks in --check mode.' echo '--verbose: Detailled informations. This can potentially leak sensitive information in the output' diff --git a/scripts/cleanup.sh b/scripts/cleanup.sh index 9f0e50cf..abe24675 100755 --- a/scripts/cleanup.sh +++ b/scripts/cleanup.sh @@ -20,6 +20,8 @@ usage(){ echo '' echo 'Available options:' echo '--ansible-extra-vars: Variable to pass as --extra-vars arguments to ansible-playbook. Make sure to escape them properly.' + echo '--ansible-path: Pass the path to the directory containing the Ansible binaries if you are not using the version of Ansible in PATH.' + echo '--python-interpreter: When using Python virtual environments Ansible may not correctly determine the Python interpreter, use this to set it manually.' echo '--previous-stable-build-number: an incremental build number that ' echo '--dry-run: Do not perform any action but run the playbooks in --check mode.' echo '--verbose: Detailled informations. This can potentially leak sensitive information in the output' diff --git a/scripts/deploy.sh b/scripts/deploy.sh index 341af1ce..66ab7fe3 100755 --- a/scripts/deploy.sh +++ b/scripts/deploy.sh @@ -20,6 +20,8 @@ usage(){ echo '' echo 'Available options:' echo '--ansible-extra-vars: Variable to pass as --extra-vars arguments to ansible-playbook. Make sure to escape them properly.' + echo '--ansible-path: Pass the path to the directory containing the Ansible binaries if you are not using the version of Ansible in PATH.' + echo '--python-interpreter: When using Python virtual environments Ansible may not correctly determine the Python interpreter, use this to set it manually.' echo '--previous-stable-build-number: an incremental build number that ' echo '--dry-run: Do not perform any action but run the playbooks in --check mode.' echo '--verbose: Detailled informations. This can potentially leak sensitive information in the output' diff --git a/scripts/revert.sh b/scripts/revert.sh index a10655a3..7b36b42b 100755 --- a/scripts/revert.sh +++ b/scripts/revert.sh @@ -20,6 +20,8 @@ usage(){ echo '' echo 'Available options:' echo '--ansible-extra-vars: Variable to pass as --extra-vars arguments to ansible-playbook. Make sure to escape them properly.' + echo '--ansible-path: Pass the path to the directory containing the Ansible binaries if you are not using the version of Ansible in PATH.' + echo '--python-interpreter: When using Python virtual environments Ansible may not correctly determine the Python interpreter, use this to set it manually.' echo '--previous-stable-build-number: an incremental build number that ' echo '--dry-run: Do not perform any action but run the playbooks in --check mode.' echo '--verbose: Detailled informations. This can potentially leak sensitive information in the output' From 5900d8df0e6afbcfc31c4e5ba083eab48a9b1e6f Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 18 Jan 2024 10:21:58 +0200 Subject: [PATCH 084/190] R67637 terminate sessions keeping loop device busy pr devel (#404) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 776b0557..b7e53d6d 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -181,6 +181,25 @@ - deploy_code.service_action == "stop" - deploy_code.services | length > 0 + - name: Get the current pts session. + ansible.builtin.shell: + cmd: "tty | sed 's/\/dev\///'" + register: deploy_pts + + - name: List sessions other than the current one. + ansible.builtin.shell: + cmd: "w -h | awk '{print $2}' | grep -v '^{{ deploy_pts.stdout }}$'" + register: active_pts_sessions + become: true + + - name: Terminate user sessions that might be keeping the loop device busy. + ansible.builtin.shell: + cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -t" + with_items: "{{ active_pts_sessions.stdout_lines }}" + when: not active_pts_sessions.stdout + become: true + failed_when: false + - name: Unmount existing SquashFS image. ansible.builtin.command: cmd: "umount {{ deploy_base_path }}" From 920c85538b8c6cd1403fa76295d40a6a508869c4 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 18 Jan 2024 10:39:03 +0200 Subject: [PATCH 085/190] R67637 terminate sessions keeping loop device busy pr devel (#406) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index b7e53d6d..5f95d092 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -183,7 +183,7 @@ - name: Get the current pts session. ansible.builtin.shell: - cmd: "tty | sed 's/\/dev\///'" + cmd: "tty | sed 's#/dev/##'" register: deploy_pts - name: List sessions other than the current one. From c952f63763e006d95b0f654e647db9a73b62416b Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 18 Jan 2024 10:49:44 +0200 Subject: [PATCH 086/190] R67637 terminate sessions keeping loop device busy pr devel (#407) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 5f95d092..ddbc696a 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -196,7 +196,7 @@ ansible.builtin.shell: cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -t" with_items: "{{ active_pts_sessions.stdout_lines }}" - when: not active_pts_sessions.stdout + when: active_pts_sessions.stdout_lines is defined become: true failed_when: false From 857f3f3e346dec74822d5e635a0df001b18d5c9a Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 18 Jan 2024 11:29:10 +0200 Subject: [PATCH 087/190] R67637 terminate sessions keeping loop device busy pr devel (#408) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index ddbc696a..a3534c75 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -194,7 +194,7 @@ - name: Terminate user sessions that might be keeping the loop device busy. ansible.builtin.shell: - cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -t" + cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -9" with_items: "{{ active_pts_sessions.stdout_lines }}" when: active_pts_sessions.stdout_lines is defined become: true From 5520b8edee8ffa5566efd2a383c6684c9d6def23 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 18 Jan 2024 11:31:25 +0200 Subject: [PATCH 088/190] R67637 terminate sessions keeping loop device busy pr devel (#409) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index a3534c75..78b77ee8 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -194,7 +194,7 @@ - name: Terminate user sessions that might be keeping the loop device busy. ansible.builtin.shell: - cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -9" + cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -9 -t" with_items: "{{ active_pts_sessions.stdout_lines }}" when: active_pts_sessions.stdout_lines is defined become: true From de15d5d5c589e850f0f6e1263474260ba113085a Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 18 Jan 2024 15:23:22 +0200 Subject: [PATCH 089/190] R67637 terminate sessions keeping loop device busy pr devel (#410) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 44 ++++++++++++++++------------- 1 file changed, 25 insertions(+), 19 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 78b77ee8..fd17a54e 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -155,6 +155,31 @@ failed_when: false register: _deploy_code_mount_check + - name: Get the current pts session. + ansible.builtin.shell: + cmd: "tty | sed 's#/dev/##'" + register: deploy_pts + + - name: List sessions other than the current one. + ansible.builtin.shell: + cmd: "w -h | awk '{print $2}' | grep -v '^{{ deploy_pts.stdout }}$'" + register: active_pts_sessions + become: true + + - name: Terminate user sessions that might be keeping the loop device busy. + ansible.builtin.shell: + cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -9 -t" + with_items: "{{ active_pts_sessions.stdout_lines }}" + when: active_pts_sessions.stdout_lines is defined + become: true + failed_when: false + + - name: Display current and terminated PTS sessions + ansible.builtin.debug: + msg: > + Current PTS session: {{ deploy_pts.stdout }} + Sessions terminated: {{ active_pts_sessions.stdout_lines | default('None') }} + - name: Reload any services that might be keeping the loop device busy. ansible.builtin.service: name: "{{ www_service }}" @@ -181,25 +206,6 @@ - deploy_code.service_action == "stop" - deploy_code.services | length > 0 - - name: Get the current pts session. - ansible.builtin.shell: - cmd: "tty | sed 's#/dev/##'" - register: deploy_pts - - - name: List sessions other than the current one. - ansible.builtin.shell: - cmd: "w -h | awk '{print $2}' | grep -v '^{{ deploy_pts.stdout }}$'" - register: active_pts_sessions - become: true - - - name: Terminate user sessions that might be keeping the loop device busy. - ansible.builtin.shell: - cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -9 -t" - with_items: "{{ active_pts_sessions.stdout_lines }}" - when: active_pts_sessions.stdout_lines is defined - become: true - failed_when: false - - name: Unmount existing SquashFS image. ansible.builtin.command: cmd: "umount {{ deploy_base_path }}" From 56e77e97af8233e1fdad84c1960a5f0c920202bd Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 19 Jan 2024 00:10:37 +0200 Subject: [PATCH 090/190] R67637 terminate sessions keeping loop device busy pr devel (#411) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index fd17a54e..78f382b4 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -160,25 +160,27 @@ cmd: "tty | sed 's#/dev/##'" register: deploy_pts - - name: List sessions other than the current one. + - name: List active sessions in {{ deploy_base_path }}. ansible.builtin.shell: - cmd: "w -h | awk '{print $2}' | grep -v '^{{ deploy_pts.stdout }}$'" - register: active_pts_sessions + cmd: "ps -eo pid,tty | awk '{print $1}' | xargs -n 1 pwdx 2>&1 | grep -v 'No such process' | grep {{ deploy_base_path }} | cut -d: -f1 | xargs -n 1 ps -o tty= -p | sort | uniq" + register: sessions_in_deploy_path become: true - - name: Terminate user sessions that might be keeping the loop device busy. - ansible.builtin.shell: - cmd: "lsof -t +D {{ deploy_base_path }} | xargs -r ps -o tty= | grep {{ item }} | xargs -r pkill -9 -t" - with_items: "{{ active_pts_sessions.stdout_lines }}" - when: active_pts_sessions.stdout_lines is defined - become: true + - name: Kill sessions except the current one. + ansible.builtin.command: + cmd: "pkill -9 -t {{ item }}" + loop: "{{ sessions_in_deploy_path.stdout_lines }}" + when: "item != deploy_pts.stdout" failed_when: false + register: kill_sessions_result + become: true - - name: Display current and terminated PTS sessions + - name: Display session info. ansible.builtin.debug: msg: > - Current PTS session: {{ deploy_pts.stdout }} - Sessions terminated: {{ active_pts_sessions.stdout_lines | default('None') }} + Deploy session: {{ deploy_pts.stdout }}. + Sessions terminated: {{ kill_sessions_result.results | selectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. + Not ended sessions: {{ kill_sessions_result.results | rejectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. - name: Reload any services that might be keeping the loop device busy. ansible.builtin.service: From daa704efa0c2d5498ee7d0102eff9b43b1ab2b9d Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 19 Jan 2024 00:41:14 +0200 Subject: [PATCH 091/190] R67637 terminate sessions keeping loop device busy pr devel (#412) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 78f382b4..f30f6918 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -180,7 +180,6 @@ msg: > Deploy session: {{ deploy_pts.stdout }}. Sessions terminated: {{ kill_sessions_result.results | selectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. - Not ended sessions: {{ kill_sessions_result.results | rejectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. - name: Reload any services that might be keeping the loop device busy. ansible.builtin.service: From 3d6c3349709ade5c7e4c1488f3bb6886a0dfee48 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 19 Jan 2024 11:15:22 +0200 Subject: [PATCH 092/190] R67637 terminate sessions keeping loop device busy pr devel (#413) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index f30f6918..02065061 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -1,18 +1,4 @@ --- -- name: Build warning. - ansible.builtin.debug: - msg: "### WARNING - if you are using a server terminal exit NOW! You have 20 seconds to comply..." - run_once: true - when: - - deploy_code.mount_type == "squashfs" - -- name: Give people 20 seconds to exit terminal sessions. - ansible.builtin.pause: - seconds: 20 - run_once: true - when: - - deploy_code.mount_type == "squashfs" - - name: Ensure codebase is writable. ansible.builtin.shell: cmd: "if [ -d {{ deploy_path_prefix }}{{ item }} ]; then chmod -R 777 {{ deploy_path_prefix }}{{ item }}; fi" @@ -160,7 +146,7 @@ cmd: "tty | sed 's#/dev/##'" register: deploy_pts - - name: List active sessions in {{ deploy_base_path }}. + - name: "Check for active sessions in {{ deploy_base_path }}." ansible.builtin.shell: cmd: "ps -eo pid,tty | awk '{print $1}' | xargs -n 1 pwdx 2>&1 | grep -v 'No such process' | grep {{ deploy_base_path }} | cut -d: -f1 | xargs -n 1 ps -o tty= -p | sort | uniq" register: sessions_in_deploy_path From 0720d53120881d3c4ccf9e3939e9d168a9ead746 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matej=20=C5=A0tajduhar?= <30931414+matej5@users.noreply.github.com> Date: Fri, 19 Jan 2024 15:37:49 +0100 Subject: [PATCH 093/190] Avoiding cache clear warning for site variable pr devel (#414) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/cache_clear/cache_clear-drupal7/tasks/main.yml | 6 +++--- roles/cache_clear/cache_clear-drupal8/tasks/main.yml | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/roles/cache_clear/cache_clear-drupal7/tasks/main.yml b/roles/cache_clear/cache_clear-drupal7/tasks/main.yml index 6f72b3a5..b68e1979 100644 --- a/roles/cache_clear/cache_clear-drupal7/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal7/tasks/main.yml @@ -1,11 +1,11 @@ --- - name: Clear Drupal 7 cache. command: - cmd: "{{ drush_bin }} -l {{ site.folder }} -y cc all" - chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cc all" + chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" with_items: "{{ drupal.sites }}" loop_control: - loop_var: site + loop_var: site_drupal run_once: true diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index 5289f254..92db6584 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,13 +1,13 @@ --- - name: Clear Drupal cache. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} -y cr" - chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" + chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" with_items: "{{ drupal.sites }}" loop_control: - loop_var: site + loop_var: site_drupal run_once: true register: _drush_output From 336b4ccfd16d166492c3dba9bc1067413bf28c1e Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Mon, 22 Jan 2024 11:17:20 +0200 Subject: [PATCH 094/190] Moving debug task to simplify diagnostics pr devel (#416) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 02065061..fae34ffb 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -152,19 +152,26 @@ register: sessions_in_deploy_path become: true + - name: Display active sessions. + ansible.builtin.debug: + msg: > + Deploy session: {{ deploy_pts.stdout }}. + Active sessions in {{ deploy_base_path }}: {{ sessions_in_deploy_path.results | selectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. + - name: Kill sessions except the current one. ansible.builtin.command: cmd: "pkill -9 -t {{ item }}" loop: "{{ sessions_in_deploy_path.stdout_lines }}" - when: "item != deploy_pts.stdout" + when: + - "item != deploy_pts.stdout" + - "item is match('^pts/\\d+$')" failed_when: false register: kill_sessions_result become: true - - name: Display session info. + - name: Display killed sessions. ansible.builtin.debug: msg: > - Deploy session: {{ deploy_pts.stdout }}. Sessions terminated: {{ kill_sessions_result.results | selectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. - name: Reload any services that might be keeping the loop device busy. From b0e4809f577b9f1afd36fb8a94b9c5fb14711521 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Mon, 22 Jan 2024 11:56:33 +0200 Subject: [PATCH 095/190] Moving debug task to simplify diagnostics pr devel (#418) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task * additional check and debug task --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index fae34ffb..83f06849 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -156,7 +156,7 @@ ansible.builtin.debug: msg: > Deploy session: {{ deploy_pts.stdout }}. - Active sessions in {{ deploy_base_path }}: {{ sessions_in_deploy_path.results | selectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. + Active sessions in {{ deploy_base_path }}: {{ sessions_in_deploy_path.stdout | default('None') }}. - name: Kill sessions except the current one. ansible.builtin.command: From 14c0dff573638915561e95a714302ccd2bcfe39f Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Mon, 22 Jan 2024 12:22:26 +0200 Subject: [PATCH 096/190] Moving debug task to simplify diagnostics pr devel (#419) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task * additional check and debug task * additional check and debug task --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 83f06849..1d1275f1 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -155,8 +155,8 @@ - name: Display active sessions. ansible.builtin.debug: msg: > - Deploy session: {{ deploy_pts.stdout }}. - Active sessions in {{ deploy_base_path }}: {{ sessions_in_deploy_path.stdout | default('None') }}. + Deploy session: {{ deploy_pts.stdout | default('Unknown') }}. + Active sessions in {{ deploy_base_path }}: {{ sessions_in_deploy_path.stdout_lines | default([]) | join(', ') | default('None') }}. - name: Kill sessions except the current one. ansible.builtin.command: @@ -172,7 +172,7 @@ - name: Display killed sessions. ansible.builtin.debug: msg: > - Sessions terminated: {{ kill_sessions_result.results | selectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. + Sessions terminated: {{ kill_sessions_result.results | selectattr('rc', 'defined') | selectattr('rc', 'equalto', 0) | map(attribute='item') | list | join(', ') | default('None') }}. - name: Reload any services that might be keeping the loop device busy. ansible.builtin.service: From 7d0aa470d5bf3735f00ae17dc05f7cfac2416367 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Mon, 22 Jan 2024 12:53:27 +0200 Subject: [PATCH 097/190] Moving debug task to simplify diagnostics pr devel (#420) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 1d1275f1..f3fc2c73 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -162,7 +162,7 @@ ansible.builtin.command: cmd: "pkill -9 -t {{ item }}" loop: "{{ sessions_in_deploy_path.stdout_lines }}" - when: + when: - "item != deploy_pts.stdout" - "item is match('^pts/\\d+$')" failed_when: false From 48e2a752ad0d03a4131eb449f5d898845eabf6a0 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:51:22 +0200 Subject: [PATCH 098/190] R65896 cache clear dir pr devel (#421) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/main.yml | 53 +++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 29 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index c054e8fb..73bac9f6 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -31,38 +31,33 @@ - cache_clear_opcache.cachetool_bin is defined - cache_clear_opcache.cachetool_bin | length > 0 -# Manipulate variables for SquashFS builds. -- name: Define image builds base path. - ansible.builtin.set_fact: - build_base_path: "/home/{{ deploy_user }}/builds/{{ project_name }}_{{ build_type }}" - when: - - deploy_code.mount_type is defined - - deploy_code.mount_type == "squashfs" -- name: Define image builds build path prefix. - ansible.builtin.set_fact: - build_path_prefix: "{{ build_base_path }}/{{ project_name }}_{{ build_type }}_build_" - when: - - deploy_code.mount_type is defined - - deploy_code.mount_type == "squashfs" -- name: Define live_symlink dest for image builds. - ansible.builtin.set_fact: - live_symlink_build_dest: "{{ live_symlink_build_dest | default('{{ build_base_path }}/live.{{ project_name }}_{{ build_type }}') }}" - when: - - deploy_code.mount_type is defined - - deploy_code.mount_type == "squashfs" -- name: Overwrite deploy and live_symlink paths if SquashFS deploy. - ansible.builtin.set_fact: - deploy_path: "{{ build_path | default('{{ build_path_prefix }}{{ build_number }}') }}" - when: - - deploy_code.mount_type is defined - - deploy_code.mount_type == "squashfs" -- name: Overwrite cachetool path if SquashFS deploy and path not provided. - ansible.builtin.set_fact: - cachetool_bin: "{{ build_base_path }}/cachetool.phar" +- name: Manipulate variables for SquashFS builds. + block: + - name: Define image builds base path. + ansible.builtin.set_fact: + build_base_path: "/home/{{ deploy_user }}/builds/{{ project_name }}_{{ build_type }}" + + - name: Define image builds build path prefix. + ansible.builtin.set_fact: + build_path_prefix: "{{ build_base_path }}/{{ project_name }}_{{ build_type }}_build_" + + - name: Define live_symlink dest for image builds. + ansible.builtin.set_fact: + live_symlink_build_dest: "{{ live_symlink_build_dest | default('{{ build_base_path }}/live.{{ project_name }}_{{ build_type }}') }}" + + - name: Overwrite deploy and live_symlink paths if SquashFS deploy. + ansible.builtin.set_fact: + deploy_path: "{{ build_path | default('{{ build_path_prefix }}{{ build_number }}') }}" + + - name: Overwrite cachetool path if SquashFS deploy and path not provided. + ansible.builtin.set_fact: + cachetool_bin: "{{ build_base_path }}/cachetool.phar" + when: + - cache_clear_opcache.cachetool_bin is not defined when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" - - cache_clear_opcache.cachetool_bin is not defined + - deploy_operation == 'deploy' # Gather last known good build directly from symlink. # This can happen: From d945ae7416d8c62cbf69873ea8ea949085ca157d Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 23 Jan 2024 19:08:15 +0200 Subject: [PATCH 099/190] R65896 cache clear dir pr devel (#423) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/main.yml | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 73bac9f6..aa5549dd 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -54,6 +54,11 @@ cachetool_bin: "{{ build_base_path }}/cachetool.phar" when: - cache_clear_opcache.cachetool_bin is not defined + + - name: Ensure build target directory exists. + ansible.builtin.file: + path: "{{ build_base_path }}" + state: directory when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" @@ -94,15 +99,6 @@ path: "{{ deploy_base_path }}" state: directory -# Make sure the build target exists. -- name: Ensure build target directory exists. - ansible.builtin.file: - path: "{{ build_base_path }}" - state: directory - when: - - deploy_code.mount_type is defined - - deploy_code.mount_type == "squashfs" - # Check for project specific init tasks. - name: Check that {{ project_type }}.yml exists. ansible.builtin.stat: From dfa66ece4fb2b22158a55c076b54af36cfb13884 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 23 Jan 2024 20:39:35 +0200 Subject: [PATCH 100/190] R65896 cache clear dir pr devel (#424) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/deploy_code/tasks/cleanup.yml | 126 +++++++++++----------------- 1 file changed, 51 insertions(+), 75 deletions(-) diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index f3fc2c73..449c1103 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -17,15 +17,6 @@ - deploy_code.perms_fix_path | length > 1 - deploy_code.mount_type != "squashfs" -- name: Ensure permissions are set on builds directory. - ansible.builtin.shell: - cmd: "if [ -d {{ build_path_prefix }}{{ item }}/{{ deploy_code.perms_fix_path }} ]; then chmod 755 {{ build_path_prefix }}{{ item }}/{{ deploy_code.perms_fix_path }}; fi" - with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - deploy_code.keep, 0] | max }} - when: - - deploy_code.perms_fix_path is defined - - deploy_code.perms_fix_path | length > 1 - - deploy_code.mount_type == "squashfs" - - name: Delete codebases from deploy directory. ansible.builtin.file: name: "{{ deploy_path_prefix }}{{ item }}" @@ -34,14 +25,6 @@ when: - deploy_code.mount_type != "squashfs" -- name: Delete codebases from builds directory. - ansible.builtin.file: - name: "{{ build_path_prefix }}{{ item }}" - state: absent - with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - deploy_code.keep, 0] | max }} - when: - - deploy_code.mount_type == "squashfs" - - name: Create a tarball of the deployed codebases. ansible.builtin.command: cmd: "tar -cvf /tmp/{{ project_name }}_{{ build_type }}_{{ build_number }}.tar --owner=0 --group=0 {{ deploy_base_path }}" @@ -51,15 +34,6 @@ - deploy_code.mount_type == "tarball" run_once: true -- name: Create a SquashFS image of the deployed codebases. - ansible.builtin.command: - cmd: "mksquashfs {{ build_base_path }} /tmp/{{ project_name }}_{{ build_type }}_{{ build_number }}.sqsh -e {{ build_base_path }}/deploy.sqsh" - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - run_once: true - - name: Create destination folder. ansible.builtin.file: path: "{{ deploy_code.mount_sync }}" @@ -79,60 +53,57 @@ - deploy_code.mount_type == "tarball" run_once: true -- name: Move SquashFS image to final destination. - ansible.builtin.command: - cmd: "mv /tmp/{{ project_name }}_{{ build_type }}_{{ build_number }}.sqsh {{ deploy_code.mount_sync }}/{{ project_name }}_{{ build_type }}.sqsh" - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - run_once: true +# Beginning of the squashFS block. +- name: Execute tasks for squashFS mount type. + block: + - name: Ensure permissions are set on builds directory. + ansible.builtin.shell: + cmd: "if [ -d {{ build_path_prefix }}{{ item }}/{{ deploy_code.perms_fix_path }} ]; then chmod 755 {{ build_path_prefix }}{{ item }}/{{ deploy_code.perms_fix_path }}; fi" + with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - deploy_code.keep, 0] | max }} + when: + - deploy_code.perms_fix_path is defined + - deploy_code.perms_fix_path | length > 1 -- name: Check if we have a SquashFS image already there. - ansible.builtin.stat: - path: "{{ build_base_path }}/deploy.sqsh" - register: _deploy_code_mount_image - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" + - name: Delete codebases from builds directory. + ansible.builtin.file: + name: "{{ build_path_prefix }}{{ item }}" + state: absent + with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - deploy_code.keep, 0] | max }} -- name: Copy previous SquashFS image to shared mount point in case of rollback. - ansible.builtin.copy: - remote_src: true - force: true - src: "{{ build_base_path }}/deploy.sqsh" - dest: "{{ deploy_code.mount_sync }}/{{ project_name }}_{{ build_type }}_previous.sqsh" - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - - _deploy_code_mount_image.stat.islnk is defined - run_once: true + - name: Create a SquashFS image of the deployed codebases. + ansible.builtin.command: + cmd: "mksquashfs {{ build_base_path }} /tmp/{{ project_name }}_{{ build_type }}_{{ build_number }}.sqsh -e {{ build_base_path }}/deploy.sqsh" + run_once: true -- name: Ensure mounted SquashFS image is deleted. - ansible.builtin.file: - path: "{{ build_base_path }}/deploy.sqsh" - state: absent - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" + - name: Move SquashFS image to final destination. + ansible.builtin.command: + cmd: "mv /tmp/{{ project_name }}_{{ build_type }}_{{ build_number }}.sqsh {{ deploy_code.mount_sync }}/{{ project_name }}_{{ build_type }}.sqsh" + run_once: true + + - name: Check if we have a SquashFS image already there. + ansible.builtin.stat: + path: "{{ build_base_path }}/deploy.sqsh" + register: _deploy_code_mount_image + + - name: Copy previous SquashFS image to shared mount point in case of rollback. + ansible.builtin.copy: + remote_src: true + force: true + src: "{{ build_base_path }}/deploy.sqsh" + dest: "{{ deploy_code.mount_sync }}/{{ project_name }}_{{ build_type }}_previous.sqsh" + when: + - _deploy_code_mount_image.stat.islnk is defined + run_once: true -- name: Copy SquashFS image to local server. - ansible.builtin.command: - cmd: "cp {{ deploy_code.mount_sync }}/{{ project_name }}_{{ build_type }}.sqsh {{ build_base_path }}/deploy.sqsh" - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" + - name: Ensure mounted SquashFS image is deleted. + ansible.builtin.file: + path: "{{ build_base_path }}/deploy.sqsh" + state: absent + + - name: Copy SquashFS image to local server. + ansible.builtin.command: + cmd: "cp {{ deploy_code.mount_sync }}/{{ project_name }}_{{ build_type }}.sqsh {{ build_base_path }}/deploy.sqsh" -- name: Unmount and remount squashfs images. - when: - - deploy_code.mount_sync is defined - - deploy_code.mount_sync | length > 1 - - deploy_code.mount_type == "squashfs" - block: - name: Check if we have a mount already. ansible.builtin.shell: cmd: "set -o pipefail && mount | grep {{ deploy_base_path }}" @@ -224,6 +195,11 @@ - _deploy_code_mount_check.rc == 0 - deploy_code.service_action == "stop" - deploy_code.services | length > 0 + when: + - deploy_code.mount_sync is defined + - deploy_code.mount_sync | length > 1 + - deploy_code.mount_type == "squashfs" +# End of the squashFS block. - name: Trigger an infrastructure rebuild. ansible.builtin.include_role: From 8a0005a0f0fd4cb0c522684e76786e70d3030fb6 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 01:15:10 +0200 Subject: [PATCH 101/190] R65896 cache clear dir pr devel (#425) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/main.yml | 21 ++++++++++++++++++- .../cache_clear-drupal8/tasks/main.yml | 2 +- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index aa5549dd..676d97bd 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -3,27 +3,35 @@ - name: Define deploy user. ansible.builtin.set_fact: deploy_user: "{{ deploy_user | default('deploy') }}" + - name: Define deploy base path. ansible.builtin.set_fact: deploy_base_path: "{{ deploy_base_path | default('/home/{{ deploy_user }}/deploy/{{ project_name }}_{{ build_type }}') }}" + - name: Define mounted directory for assets. ansible.builtin.set_fact: deploy_assets_base_path: "{{ deploy_assets_base_path | default('/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/assets') }}" + - name: Define webroot. ansible.builtin.set_fact: webroot: "{{ webroot | default('web') }}" + - name: Define build deploy path prefix. ansible.builtin.set_fact: deploy_path_prefix: "{{ deploy_base_path }}/{{ project_name }}_{{ build_type }}_build_" + - name: Define build deploy path. ansible.builtin.set_fact: deploy_path: "{{ deploy_path | default('{{ deploy_path_prefix }}{{ build_number }}') }}" + - name: Define live_symlink dest. ansible.builtin.set_fact: live_symlink_dest: "{{ live_symlink_dest | default('{{ deploy_base_path }}/live.{{ project_name }}_{{ build_type }}') }}" + - name: Define opcache cachetool path. ansible.builtin.set_fact: cachetool_bin: "{{ deploy_base_path }}/cachetool.phar" + - name: Set opcache cachetool path from variable if provided. ansible.builtin.set_fact: cachetool_bin: "{{ cache_clear_opcache.cachetool_bin }}" @@ -31,6 +39,18 @@ - cache_clear_opcache.cachetool_bin is defined - cache_clear_opcache.cachetool_bin | length > 0 +- name: Define path to clear cache. + ansible.builtin.set_fact: + cache_clear_base_path: "{{ deploy_path }}" + when: + - deploy_operation != 'cleanup' + +- name: Define path to clear cache. + ansible.builtin.set_fact: + cache_clear_base_path: "{{ cache_clear_base_path | default('{{ deploy_path_prefix }}{{ build_number }}') }}" + when: + - deploy_operation == 'cleanup' + - name: Manipulate variables for SquashFS builds. block: - name: Define image builds base path. @@ -62,7 +82,6 @@ when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" - - deploy_operation == 'deploy' # Gather last known good build directly from symlink. # This can happen: diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index 92db6584..e16ab252 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -2,7 +2,7 @@ - name: Clear Drupal cache. ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" - chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" + chdir: "{{ cache_clear_base_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" with_items: "{{ drupal.sites }}" From 7f6ef30c7aaf914df69a2a6e180d4c5aab6c9c75 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 01:36:18 +0200 Subject: [PATCH 102/190] R65896 cache clear dir pr devel (#426) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/main.yml | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 676d97bd..21b48dc3 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -39,18 +39,6 @@ - cache_clear_opcache.cachetool_bin is defined - cache_clear_opcache.cachetool_bin | length > 0 -- name: Define path to clear cache. - ansible.builtin.set_fact: - cache_clear_base_path: "{{ deploy_path }}" - when: - - deploy_operation != 'cleanup' - -- name: Define path to clear cache. - ansible.builtin.set_fact: - cache_clear_base_path: "{{ cache_clear_base_path | default('{{ deploy_path_prefix }}{{ build_number }}') }}" - when: - - deploy_operation == 'cleanup' - - name: Manipulate variables for SquashFS builds. block: - name: Define image builds base path. @@ -83,6 +71,18 @@ - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" +- name: Define path to clear cache for the deploy step. + ansible.builtin.set_fact: + cache_clear_base_path: "{{ deploy_path }}" + when: + - deploy_operation != 'cleanup' + +- name: Define path to clear cache for the cleanup step. + ansible.builtin.set_fact: + cache_clear_base_path: "{{ cache_clear_base_path | default('{{ deploy_path_prefix }}{{ build_number }}') }}" + when: + - deploy_operation == 'cleanup' + # Gather last known good build directly from symlink. # This can happen: # - when the first builds failed, From c6212213a23153643089931c7da2eb7260664019 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 11:23:16 +0200 Subject: [PATCH 103/190] R65896 cache clear dir pr devel (#427) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../cache_clear-drupal8/tasks/main.yml | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index e16ab252..f8ba1228 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,5 +1,5 @@ --- -- name: Clear Drupal cache. +- name: Clear Drupal cache, drush is in site build dir. ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" chdir: "{{ cache_clear_base_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" @@ -10,6 +10,22 @@ loop_var: site_drupal run_once: true register: _drush_output + when: + - deploy_operation != 'cleanup' + +- name: Clear Drupal cache, drush is in site live dir. + ansible.builtin.command: + cmd: "{{ drush_live_bin }} -l {{ site_drupal.folder }} -y cr" + chdir: "{{ cache_clear_base_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" + become: "{{ 'no' if www_user == deploy_user else 'yes' }}" + become_user: "{{ www_user }}" + with_items: "{{ drupal.sites }}" + loop_control: + loop_var: site_drupal + run_once: true + register: _drush_output + when: + - deploy_operation == 'cleanup' - name: Show drush output. ansible.builtin.debug: From 8f9f72f6643a7091f1fca9fb02f76d3b4eb2f36e Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 12:20:06 +0200 Subject: [PATCH 104/190] R65896 cache clear dir pr devel (#428) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/drupal8.yml | 2 +- .../cache_clear-drupal8/tasks/main.yml | 22 +++++-------------- 2 files changed, 7 insertions(+), 17 deletions(-) diff --git a/roles/_init/tasks/drupal8.yml b/roles/_init/tasks/drupal8.yml index 3792c489..478b4426 100644 --- a/roles/_init/tasks/drupal8.yml +++ b/roles/_init/tasks/drupal8.yml @@ -5,4 +5,4 @@ - name: Define path to drush for currently live build. set_fact: - drush_live_bin: "{{ drush_live_bin | default('{{ live_symlink_dest }}/vendor/bin/drush') }}" \ No newline at end of file + drush_live_bin: "{{ drush_live_bin | default('{{ live_symlink_dest }}/vendor/bin/drush') }}" diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index f8ba1228..a4a4203a 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,21 +1,13 @@ --- -- name: Clear Drupal cache, drush is in site build dir. - ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" - chdir: "{{ cache_clear_base_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" - become: "{{ 'no' if www_user == deploy_user else 'yes' }}" - become_user: "{{ www_user }}" - with_items: "{{ drupal.sites }}" - loop_control: - loop_var: site_drupal - run_once: true - register: _drush_output +- name: Override path to drush_bin on the cleanup step. + set_fact: + drush_bin: "{{ drush_live_bin }}" when: - - deploy_operation != 'cleanup' + - deploy_operation == 'cleanup' -- name: Clear Drupal cache, drush is in site live dir. +- name: Clear Drupal cache. ansible.builtin.command: - cmd: "{{ drush_live_bin }} -l {{ site_drupal.folder }} -y cr" + cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" chdir: "{{ cache_clear_base_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" @@ -24,8 +16,6 @@ loop_var: site_drupal run_once: true register: _drush_output - when: - - deploy_operation == 'cleanup' - name: Show drush output. ansible.builtin.debug: From dec5f6bdbcfea535d3b65d0de74ba5ae415ea95b Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 12:26:01 +0200 Subject: [PATCH 105/190] R65896 cache clear dir pr devel (#429) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/main.yml | 6 ++++++ roles/cache_clear/cache_clear-drupal8/tasks/main.yml | 6 ------ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 21b48dc3..cecc89e2 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -67,6 +67,12 @@ ansible.builtin.file: path: "{{ build_base_path }}" state: directory + + - name: Override path to drush_bin on the cleanup step. + set_fact: + drush_bin: "{{ drush_live_bin }}" + when: + - deploy_operation == 'cleanup' when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index a4a4203a..e16ab252 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,10 +1,4 @@ --- -- name: Override path to drush_bin on the cleanup step. - set_fact: - drush_bin: "{{ drush_live_bin }}" - when: - - deploy_operation == 'cleanup' - - name: Clear Drupal cache. ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" From 7bbdd3d799a33dc27a23119e88350c40da6d2360 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 12:43:59 +0200 Subject: [PATCH 106/190] R65896 cache clear dir pr devel (#430) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/drupal8.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/roles/_init/tasks/drupal8.yml b/roles/_init/tasks/drupal8.yml index 478b4426..19e7204d 100644 --- a/roles/_init/tasks/drupal8.yml +++ b/roles/_init/tasks/drupal8.yml @@ -6,3 +6,9 @@ - name: Define path to drush for currently live build. set_fact: drush_live_bin: "{{ drush_live_bin | default('{{ live_symlink_dest }}/vendor/bin/drush') }}" + +- name: Override path to drush_bin on the cleanup step. + set_fact: + drush_bin: "{{ drush_live_bin }}" + when: + - deploy_operation == 'cleanup' From a1641f2b2b1995d6741e8f023152133717689c9a Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 13:26:41 +0200 Subject: [PATCH 107/190] R65896 cache clear dir pr devel (#431) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/drupal8.yml | 6 ------ roles/_init/tasks/main.yml | 6 ------ roles/cache_clear/cache_clear-drupal8/tasks/main.yml | 6 ++++++ 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/roles/_init/tasks/drupal8.yml b/roles/_init/tasks/drupal8.yml index 19e7204d..478b4426 100644 --- a/roles/_init/tasks/drupal8.yml +++ b/roles/_init/tasks/drupal8.yml @@ -6,9 +6,3 @@ - name: Define path to drush for currently live build. set_fact: drush_live_bin: "{{ drush_live_bin | default('{{ live_symlink_dest }}/vendor/bin/drush') }}" - -- name: Override path to drush_bin on the cleanup step. - set_fact: - drush_bin: "{{ drush_live_bin }}" - when: - - deploy_operation == 'cleanup' diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index cecc89e2..21b48dc3 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -67,12 +67,6 @@ ansible.builtin.file: path: "{{ build_base_path }}" state: directory - - - name: Override path to drush_bin on the cleanup step. - set_fact: - drush_bin: "{{ drush_live_bin }}" - when: - - deploy_operation == 'cleanup' when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index e16ab252..a4a4203a 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,4 +1,10 @@ --- +- name: Override path to drush_bin on the cleanup step. + set_fact: + drush_bin: "{{ drush_live_bin }}" + when: + - deploy_operation == 'cleanup' + - name: Clear Drupal cache. ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" From 7b38f182c12872566c2ad648ef7b9791dbb55785 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 24 Jan 2024 14:15:37 +0200 Subject: [PATCH 108/190] R65896 cache clear dir pr devel (#432) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/main.yml | 12 ------------ .../cache_clear-drupal8/tasks/main.yml | 15 +++++++++++---- 2 files changed, 11 insertions(+), 16 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 21b48dc3..225e9b60 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -71,18 +71,6 @@ - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" -- name: Define path to clear cache for the deploy step. - ansible.builtin.set_fact: - cache_clear_base_path: "{{ deploy_path }}" - when: - - deploy_operation != 'cleanup' - -- name: Define path to clear cache for the cleanup step. - ansible.builtin.set_fact: - cache_clear_base_path: "{{ cache_clear_base_path | default('{{ deploy_path_prefix }}{{ build_number }}') }}" - when: - - deploy_operation == 'cleanup' - # Gather last known good build directly from symlink. # This can happen: # - when the first builds failed, diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index a4a4203a..1fc9d2f2 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -1,14 +1,21 @@ --- -- name: Override path to drush_bin on the cleanup step. - set_fact: - drush_bin: "{{ drush_live_bin }}" +- name: Override paths for squashfs build cleanup step. + block: + - name: Override path to drush binary. + set_fact: + drush_bin: "{{ drush_live_bin }}" + - name: Override deploy_path. + set_fact: + deploy_path: "{{ live_symlink_dest }}" when: + - deploy_code.mount_type is defined + - deploy_code.mount_type == "squashfs" - deploy_operation == 'cleanup' - name: Clear Drupal cache. ansible.builtin.command: cmd: "{{ drush_bin }} -l {{ site_drupal.folder }} -y cr" - chdir: "{{ cache_clear_base_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" + chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" with_items: "{{ drupal.sites }}" From 43a66e4156e07bce6c901d0e14053ab59fe60eea Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 30 Jan 2024 00:45:02 +0200 Subject: [PATCH 109/190] R65896 cache container table truncate pr devel (#433) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../cache_clear-drupal8/tasks/main.yml | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index 1fc9d2f2..c95c0087 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -4,9 +4,30 @@ - name: Override path to drush binary. set_fact: drush_bin: "{{ drush_live_bin }}" + - name: Override deploy_path. set_fact: deploy_path: "{{ live_symlink_dest }}" + + - name: Truncate table cache_container. + ansible.builtin.command: + cmd: "{{ drush_bin }} sql-query 'TRUNCATE TABLE cache_container'" + chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" + become: "{{ 'no' if www_user == deploy_user else 'yes' }}" + become_user: "{{ www_user }}" + with_items: "{{ drupal.sites }}" + loop_control: + loop_var: site_drupal + run_once: true + register: _drush_output + when: + - drupal.truncate_cache_container_table is defined + - drupal.truncate_cache_container_table + + - name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: drupal.drush_verbose_output when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" From d6050d11a7a8ff0583d4b27fef8913acf7594044 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 30 Jan 2024 00:56:10 +0200 Subject: [PATCH 110/190] R65896 cache container table truncate pr devel (#435) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/cache_clear/cache_clear-drupal8/tasks/main.yml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index c95c0087..a23a4652 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -20,9 +20,7 @@ loop_var: site_drupal run_once: true register: _drush_output - when: - - drupal.truncate_cache_container_table is defined - - drupal.truncate_cache_container_table + when: drupal.truncate_cache_table - name: Show drush output. ansible.builtin.debug: From 8e6b462b524134dd863adbd656c2140d297554cf Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 30 Jan 2024 09:54:50 +0200 Subject: [PATCH 111/190] R65896 cache container table truncate pr devel (#436) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/cache_clear/cache_clear-drupal8/tasks/main.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index a23a4652..9a02f48d 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -25,7 +25,9 @@ - name: Show drush output. ansible.builtin.debug: msg: "{{ _drush_output }}" - when: drupal.drush_verbose_output + when: + - drupal.drush_verbose_output + - drupal.truncate_cache_table when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" From 947182ab21eefafe0d27212a739a2fcbb15e0f0c Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 30 Jan 2024 10:08:16 +0200 Subject: [PATCH 112/190] R65896 cache container table truncate pr devel (#437) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/defaults/main.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/roles/_init/defaults/main.yml b/roles/_init/defaults/main.yml index 5fcee089..2e4eb0a8 100644 --- a/roles/_init/defaults/main.yml +++ b/roles/_init/defaults/main.yml @@ -7,6 +7,8 @@ deploy_user: "deploy" _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" drupal: drush_verbose_output: false + # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error. + truncate_cache_table: false sites: - folder: "default" public_files: "sites/default/files" From af2c203bd608f2d7e7ce04ddef7463bf5244286a Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 31 Jan 2024 16:38:51 +0200 Subject: [PATCH 113/190] R65896 cache container table truncate change tasks order pr devel (#438) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../cache_clear-drupal8/tasks/main.yml | 45 ++++++++++--------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index 9a02f48d..a0f960da 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -8,26 +8,6 @@ - name: Override deploy_path. set_fact: deploy_path: "{{ live_symlink_dest }}" - - - name: Truncate table cache_container. - ansible.builtin.command: - cmd: "{{ drush_bin }} sql-query 'TRUNCATE TABLE cache_container'" - chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" - become: "{{ 'no' if www_user == deploy_user else 'yes' }}" - become_user: "{{ www_user }}" - with_items: "{{ drupal.sites }}" - loop_control: - loop_var: site_drupal - run_once: true - register: _drush_output - when: drupal.truncate_cache_table - - - name: Show drush output. - ansible.builtin.debug: - msg: "{{ _drush_output }}" - when: - - drupal.drush_verbose_output - - drupal.truncate_cache_table when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" @@ -49,3 +29,28 @@ ansible.builtin.debug: msg: "{{ _drush_output }}" when: drupal.drush_verbose_output + +- name: Truncate SQL table 'cache_container'. + block: + - name: Truncate table 'cache_container'. + ansible.builtin.command: + cmd: "{{ drush_bin }} sql-query 'TRUNCATE TABLE cache_container'" + chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site_drupal.folder }}" + become: "{{ 'no' if www_user == deploy_user else 'yes' }}" + become_user: "{{ www_user }}" + with_items: "{{ drupal.sites }}" + loop_control: + loop_var: site_drupal + run_once: true + register: _drush_output + + - name: Show drush output. + ansible.builtin.debug: + msg: "{{ _drush_output }}" + when: + - drupal.drush_verbose_output + when: + - drupal.truncate_cache_table + - deploy_code.mount_type is defined + - deploy_code.mount_type == "squashfs" + - deploy_operation == 'cleanup' From a28bfa82e2748b6d19bf83aadbb399e165f539c7 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 31 Jan 2024 19:08:42 +0100 Subject: [PATCH 114/190] Bug fixes pr devel (#440) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Moving installation of cachetool to application includes, where they are PHP apps. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- roles/_init/tasks/drupal7.yml | 5 +++++ roles/_init/tasks/drupal8.yml | 5 +++++ roles/_init/tasks/main.yml | 5 ----- roles/_init/tasks/matomo.yml | 6 ++++-- roles/_init/tasks/mautic.yml | 6 ++++-- 5 files changed, 18 insertions(+), 9 deletions(-) diff --git a/roles/_init/tasks/drupal7.yml b/roles/_init/tasks/drupal7.yml index b1781c1b..3053c5c4 100644 --- a/roles/_init/tasks/drupal7.yml +++ b/roles/_init/tasks/drupal7.yml @@ -10,3 +10,8 @@ - name: Ensure we have a drush binary. import_role: name: cli/drush + +- name: Ensure we have a cachetool binary. + ansible.builtin.import_role: + name: cli/cachetool + when: install_php_cachetool diff --git a/roles/_init/tasks/drupal8.yml b/roles/_init/tasks/drupal8.yml index 478b4426..7c64719e 100644 --- a/roles/_init/tasks/drupal8.yml +++ b/roles/_init/tasks/drupal8.yml @@ -6,3 +6,8 @@ - name: Define path to drush for currently live build. set_fact: drush_live_bin: "{{ drush_live_bin | default('{{ live_symlink_dest }}/vendor/bin/drush') }}" + +- name: Ensure we have a cachetool binary. + ansible.builtin.import_role: + name: cli/cachetool + when: install_php_cachetool diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 225e9b60..f57be10e 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -118,8 +118,3 @@ ansible.builtin.include_tasks: "{{ project_type }}.yml" when: - _project_type_task_result.stat.exists - -- name: Ensure we have a cachetool binary. - ansible.builtin.import_role: - name: cli/cachetool - when: install_php_cachetool diff --git a/roles/_init/tasks/matomo.yml b/roles/_init/tasks/matomo.yml index 6f34c92f..32f693a1 100644 --- a/roles/_init/tasks/matomo.yml +++ b/roles/_init/tasks/matomo.yml @@ -1,3 +1,5 @@ --- - -# Nothing to do here. \ No newline at end of file +- name: Ensure we have a cachetool binary. + ansible.builtin.import_role: + name: cli/cachetool + when: install_php_cachetool diff --git a/roles/_init/tasks/mautic.yml b/roles/_init/tasks/mautic.yml index 6f34c92f..32f693a1 100644 --- a/roles/_init/tasks/mautic.yml +++ b/roles/_init/tasks/mautic.yml @@ -1,3 +1,5 @@ --- - -# Nothing to do here. \ No newline at end of file +- name: Ensure we have a cachetool binary. + ansible.builtin.import_role: + name: cli/cachetool + when: install_php_cachetool From 74133a70f9d9d895c296b2f953495e4891760486 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 8 Feb 2024 14:12:15 +0100 Subject: [PATCH 115/190] Db replica support pr devel (#442) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- docs/roles/_init.md | 2 ++ .../database_backup/database_backup-mysql.md | 19 ++++++++++++++++++- roles/_init/README.md | 2 ++ .../database_backup-mysql/README.md | 19 ++++++++++++++++++- .../database_backup-mysql/defaults/main.yml | 4 +++- .../tasks/deploy-none.yml | 7 +++++++ 6 files changed, 50 insertions(+), 3 deletions(-) diff --git a/docs/roles/_init.md b/docs/roles/_init.md index d561aac2..98179138 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -16,6 +16,8 @@ deploy_user: "deploy" _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" drupal: drush_verbose_output: false + # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error. + truncate_cache_table: false sites: - folder: "default" public_files: "sites/default/files" diff --git a/docs/roles/database_backup/database_backup-mysql.md b/docs/roles/database_backup/database_backup-mysql.md index 1200f97f..b2c79799 100644 --- a/docs/roles/database_backup/database_backup-mysql.md +++ b/docs/roles/database_backup/database_backup-mysql.md @@ -1,5 +1,20 @@ # MySQL backups Generate MySQL backups for each build. + +## Replicas +If you are using a read only replica in your application and you need to add it to `databases` in order to have access to the credentials for your app settings, be sure to set the database up in a similar way to this: + +```yaml +mysql_backup: + databases: + - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" + user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" + credentials_file: "/home/{{ deploy_user }}/.mysql.creds" + handling: none # prevents the replica from being backed up + is_replica: true # tells ce-deploy we are working with a replica, so it will implement a pause + pause_seconds: 30 # duration of the pause in seconds +``` + ## Default variables ```yaml @@ -27,7 +42,9 @@ mysql_backup: - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" - #handling: static # optional override to the main handling method on a per database basis + #handling: none # optional override to the main handling method on a per database basis - must be 'none' for replicas + #is_replica: true # tell ce-deploy this database is a replica + #pause_seconds: 30 # how long to allow for replication to catch up, required if 'is_replica' is set to 'true' ``` diff --git a/roles/_init/README.md b/roles/_init/README.md index d561aac2..98179138 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -16,6 +16,8 @@ deploy_user: "deploy" _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" drupal: drush_verbose_output: false + # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error. + truncate_cache_table: false sites: - folder: "default" public_files: "sites/default/files" diff --git a/roles/database_backup/database_backup-mysql/README.md b/roles/database_backup/database_backup-mysql/README.md index 1200f97f..b2c79799 100644 --- a/roles/database_backup/database_backup-mysql/README.md +++ b/roles/database_backup/database_backup-mysql/README.md @@ -1,5 +1,20 @@ # MySQL backups Generate MySQL backups for each build. + +## Replicas +If you are using a read only replica in your application and you need to add it to `databases` in order to have access to the credentials for your app settings, be sure to set the database up in a similar way to this: + +```yaml +mysql_backup: + databases: + - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" + user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" + credentials_file: "/home/{{ deploy_user }}/.mysql.creds" + handling: none # prevents the replica from being backed up + is_replica: true # tells ce-deploy we are working with a replica, so it will implement a pause + pause_seconds: 30 # duration of the pause in seconds +``` + ## Default variables ```yaml @@ -27,7 +42,9 @@ mysql_backup: - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" - #handling: static # optional override to the main handling method on a per database basis + #handling: none # optional override to the main handling method on a per database basis - must be 'none' for replicas + #is_replica: true # tell ce-deploy this database is a replica + #pause_seconds: 30 # how long to allow for replication to catch up, required if 'is_replica' is set to 'true' ``` diff --git a/roles/database_backup/database_backup-mysql/defaults/main.yml b/roles/database_backup/database_backup-mysql/defaults/main.yml index 1091f8d7..23fa413a 100644 --- a/roles/database_backup/database_backup-mysql/defaults/main.yml +++ b/roles/database_backup/database_backup-mysql/defaults/main.yml @@ -22,4 +22,6 @@ mysql_backup: - database: "{{ (project_name + '_' + build_type) | regex_replace('-', '_') }}" # avoid hyphens in MySQL database names user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" - #handling: static # optional override to the main handling method on a per database basis + #handling: none # optional override to the main handling method on a per database basis - must be 'none' for replicas + #is_replica: true # tell ce-deploy this database is a replica + #pause_seconds: 30 # how long to allow for replication to catch up, required if 'is_replica' is set to 'true' diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml index 4c4b1624..aa8c7317 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml @@ -4,6 +4,13 @@ - ansible.builtin.set_fact: _mysql_previous_build_database_name: "{{ database.database }}" +- name: Pause to allow replication to catch up. + ansible.builtin.pause: + seconds: "{{ database.pause_seconds }}" + when: + - database.is_replica is defined + - database.is_replica + - name: Check if the database exists already. ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_build_database_name }}" register: _build_db_status From 27dd509bf8550af1444ca0562630fe7f9485466d Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 22 Feb 2024 13:08:25 +0100 Subject: [PATCH 116/190] Db replica support pr devel (#444) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../database_backup-mysql/tasks/cleanup-rolling.yml | 2 ++ .../database_backup-mysql/tasks/deploy-dump.yml | 4 +++- .../database_backup-mysql/tasks/deploy-none.yml | 11 +++-------- .../database_backup-mysql/tasks/deploy-rolling.yml | 2 ++ .../database_backup-mysql/tasks/revert-dump.yml | 4 +++- 5 files changed, 13 insertions(+), 10 deletions(-) diff --git a/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml index 374e65ca..6d132cd5 100644 --- a/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml @@ -8,3 +8,5 @@ login_password: "{{ _mysql_password }}" with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - mysql_backup.keep, 0] | max }} run_once: true + when: + - database.is_replica is not defined diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml index 49bda27e..32108e0a 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml @@ -20,7 +20,9 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" - when: previous_build_number == 0 or _build_db_status.rc == 1 + when: + - previous_build_number == 0 or _build_db_status.rc == 1 + - database.is_replica is not defined run_once: true - name: Ensure the dump directory exists. diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml index aa8c7317..fadc79bb 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml @@ -4,13 +4,6 @@ - ansible.builtin.set_fact: _mysql_previous_build_database_name: "{{ database.database }}" -- name: Pause to allow replication to catch up. - ansible.builtin.pause: - seconds: "{{ database.pause_seconds }}" - when: - - database.is_replica is defined - - database.is_replica - - name: Check if the database exists already. ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_build_database_name }}" register: _build_db_status @@ -27,5 +20,7 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" - when: previous_build_number == 0 or _build_db_status.rc == 1 + when: + - previous_build_number == 0 or _build_db_status.rc == 1 + - database.is_replica is not defined run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index 30fbaa62..38e82fa4 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -20,6 +20,8 @@ config_file: "{{ database.credentials_file }}" config_overrides_defaults: true run_once: true + when: + - database.is_replica is not defined - name: Check if the previous database exists. ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_previous_build_database_name }}" diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index 6040c73a..e99a8bda 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -19,7 +19,9 @@ args: executable: /bin/bash run_once: true - when: previous_build_number > 0 + when: + - previous_build_number > 0 + - database.is_replica is not defined - name: Delete unpacked dump file. ansible.builtin.file: From 893acd2bd61a4e5e7b5b1f3c17e1e7c75c91643f Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 23 Feb 2024 17:02:46 +0100 Subject: [PATCH 117/190] Db replica support pr devel (#446) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar --- .../database_backup/database_backup-mysql.md | 4 +--- .../cron_database_backup-mysql/tasks/main.yml | 4 ++-- .../tasks/setup.yml | 22 ++++++++++--------- .../database_backup-mysql/README.md | 4 +--- .../database_backup-mysql/defaults/main.yml | 3 +-- .../tasks/cleanup-rolling.yml | 2 -- .../database_backup-mysql/tasks/cleanup.yml | 5 ++++- .../tasks/deploy-dump.yml | 4 +--- .../tasks/deploy-none.yml | 4 +--- .../tasks/deploy-rolling.yml | 2 -- .../database_backup-mysql/tasks/deploy.yml | 10 +++++++-- .../tasks/revert-dump.yml | 4 +--- .../database_backup-mysql/tasks/revert.yml | 1 + 13 files changed, 33 insertions(+), 36 deletions(-) diff --git a/docs/roles/database_backup/database_backup-mysql.md b/docs/roles/database_backup/database_backup-mysql.md index b2c79799..2766a4f3 100644 --- a/docs/roles/database_backup/database_backup-mysql.md +++ b/docs/roles/database_backup/database_backup-mysql.md @@ -12,7 +12,6 @@ mysql_backup: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" handling: none # prevents the replica from being backed up is_replica: true # tells ce-deploy we are working with a replica, so it will implement a pause - pause_seconds: 30 # duration of the pause in seconds ``` @@ -43,8 +42,7 @@ mysql_backup: user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" #handling: none # optional override to the main handling method on a per database basis - must be 'none' for replicas - #is_replica: true # tell ce-deploy this database is a replica - #pause_seconds: 30 # how long to allow for replication to catch up, required if 'is_replica' is set to 'true' + #is_replica: true # tell ce-deploy this database is a replica - can only be true, remove/comment out if not required ``` diff --git a/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/main.yml b/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/main.yml index ffd14157..e87cad12 100644 --- a/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/main.yml +++ b/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/main.yml @@ -4,7 +4,7 @@ # add a keep mechanism for backup scripts, like for the dumps themselves. # Nice to have more than anything. - name: Setup PATH in crontab. - cron: + ansible.builtin.cron: name: PATH env: true job: "/usr/bin:/usr/local/bin:/bin:/home/{{ deploy_user }}/.bin" @@ -13,7 +13,7 @@ - drupal.defer is defined - drupal.defer -- include_tasks: setup.yml +- ansible.builtin.include_tasks: setup.yml vars: database: database with_items: "{{ build_databases }}" diff --git a/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/setup.yml b/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/setup.yml index d09fbbac..f12301da 100644 --- a/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/setup.yml +++ b/roles/cron/cron_database_backup/cron_database_backup-mysql/tasks/setup.yml @@ -1,39 +1,40 @@ # Probably not worth adding more granularity (weeks, months) for backups, should be at least daily? -- set_fact: +- ansible.builtin.set_fact: _cron_mysql_backup_random_minute: "{{ 59 | random }}" -- set_fact: +- ansible.builtin.set_fact: _cron_mysql_backup_minute: "{{ database.original.backup.minute | default(_cron_mysql_backup_random_minute) }}" -- set_fact: +- ansible.builtin.set_fact: _cron_mysql_backup_hour: "{{ database.original.backup.hour | default(0) }}" -- set_fact: +- ansible.builtin.set_fact: _cron_mysql_backup_keep: "{{ database.original.backup.keep | default(10) }}" - name: Ensure backup directory exists. - file: + ansible.builtin.file: path: "{{ cron_mysql_backup.dumps_directory }}/{{ database.host }}" state: directory mode: 0700 recurse: true - name: Ensure cron directory exists. - file: + ansible.builtin.file: path: "/home/{{ deploy_user }}/cron/{{ project_name }}_{{ build_type }}" state: directory mode: 0700 recurse: true - name: Create backup script. - template: + ansible.builtin.template: src: "regular-backups.sh.j2" dest: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/{{ database.host }}-{{ database.original.database }}-regular-backups.sh" mode: 0700 + when: database.is_replica is not defined - name: Define backup cron job command. - set_fact: + ansible.builtin.set_fact: _backup_cron_job_command: "/bin/sh /home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/{{ database.host }}-{{ database.original.database }}-regular-backups.sh" - name: Define backup cron job command if deferred (ASG). - set_fact: + ansible.builtin.set_fact: _backup_cron_job_command: "cd {{ _ce_deploy_base_dir }} && {{ _ce_deploy_ansible_location }} {{ drupal.defer_target }} -m shell -a \"{{ _backup_cron_job_command }}\"" when: - drupal.defer is defined @@ -42,9 +43,10 @@ - drupal.defer_target | length > 0 - name: Setup regular backup for MySQL. - cron: + ansible.builtin.cron: name: "cron_mysql_{{ database.host }}_{{ database.original.database }}" minute: "{{ _cron_mysql_backup_minute }}" hour: "{{ _cron_mysql_backup_hour }}" job: "{{ _backup_cron_job_command }}" delegate_to: "{{ 'localhost' if drupal.defer is defined and drupal.defer else inventory_hostname }}" + when: database.is_replica is not defined diff --git a/roles/database_backup/database_backup-mysql/README.md b/roles/database_backup/database_backup-mysql/README.md index b2c79799..2766a4f3 100644 --- a/roles/database_backup/database_backup-mysql/README.md +++ b/roles/database_backup/database_backup-mysql/README.md @@ -12,7 +12,6 @@ mysql_backup: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" handling: none # prevents the replica from being backed up is_replica: true # tells ce-deploy we are working with a replica, so it will implement a pause - pause_seconds: 30 # duration of the pause in seconds ``` @@ -43,8 +42,7 @@ mysql_backup: user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" #handling: none # optional override to the main handling method on a per database basis - must be 'none' for replicas - #is_replica: true # tell ce-deploy this database is a replica - #pause_seconds: 30 # how long to allow for replication to catch up, required if 'is_replica' is set to 'true' + #is_replica: true # tell ce-deploy this database is a replica - can only be true, remove/comment out if not required ``` diff --git a/roles/database_backup/database_backup-mysql/defaults/main.yml b/roles/database_backup/database_backup-mysql/defaults/main.yml index 23fa413a..e7351d89 100644 --- a/roles/database_backup/database_backup-mysql/defaults/main.yml +++ b/roles/database_backup/database_backup-mysql/defaults/main.yml @@ -23,5 +23,4 @@ mysql_backup: user: "{{ (project_name + '_' + build_type) | truncate(32, true, '', 0) }}" # 32 char limit credentials_file: "/home/{{ deploy_user }}/.mysql.creds" #handling: none # optional override to the main handling method on a per database basis - must be 'none' for replicas - #is_replica: true # tell ce-deploy this database is a replica - #pause_seconds: 30 # how long to allow for replication to catch up, required if 'is_replica' is set to 'true' + #is_replica: true # tell ce-deploy this database is a replica - can only be true, remove/comment out if not required diff --git a/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml index 6d132cd5..374e65ca 100644 --- a/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml @@ -8,5 +8,3 @@ login_password: "{{ _mysql_password }}" with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - mysql_backup.keep, 0] | max }} run_once: true - when: - - database.is_replica is not defined diff --git a/roles/database_backup/database_backup-mysql/tasks/cleanup.yml b/roles/database_backup/database_backup-mysql/tasks/cleanup.yml index 89cbf171..23654247 100644 --- a/roles/database_backup/database_backup-mysql/tasks/cleanup.yml +++ b/roles/database_backup/database_backup-mysql/tasks/cleanup.yml @@ -14,6 +14,7 @@ _mysql_password: "{{ lookup('ini', 'password section=client file={{ _ce_deploy_build_dir }}/mysql_backup_credentials.ini') }}" - ansible.builtin.include_tasks: "cleanup-{{ mysql_backup.handling }}.yml" + when: database.is_replica is not defined - name: Delete mysql users. community.mysql.mysql_user: @@ -24,5 +25,7 @@ login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - mysql_backup.keep, 0] | max }} - when: mysql_backup.credentials_handling == 'rotate' + when: + - mysql_backup.credentials_handling == 'rotate' + - database.is_replica is not defined run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml index 32108e0a..49bda27e 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml @@ -20,9 +20,7 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" - when: - - previous_build_number == 0 or _build_db_status.rc == 1 - - database.is_replica is not defined + when: previous_build_number == 0 or _build_db_status.rc == 1 run_once: true - name: Ensure the dump directory exists. diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml index fadc79bb..4c4b1624 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml @@ -20,7 +20,5 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" - when: - - previous_build_number == 0 or _build_db_status.rc == 1 - - database.is_replica is not defined + when: previous_build_number == 0 or _build_db_status.rc == 1 run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index 38e82fa4..30fbaa62 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -20,8 +20,6 @@ config_file: "{{ database.credentials_file }}" config_overrides_defaults: true run_once: true - when: - - database.is_replica is not defined - name: Check if the previous database exists. ansible.builtin.shell: "set -o pipefail && mysql --defaults-extra-file={{ database.credentials_file }} -e 'SHOW DATABASES;' | grep -w {{ _mysql_previous_build_database_name }}" diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy.yml b/roles/database_backup/database_backup-mysql/tasks/deploy.yml index 8d93c118..f8ac6f62 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy.yml @@ -97,8 +97,10 @@ when: - database.handling is defined - database.handling | length > 0 + - name: Execute backup tasks. ansible.builtin.include_tasks: "deploy-{{ _mysql_handling }}.yml" + when: database.is_replica is not defined # We append privileges instead of replacing, # to allow this role to be looped over, @@ -107,12 +109,16 @@ # As of MySQL 8.0 the GRANT operation has no password option, you must CREATE your user first. - name: Create/update mysql user for TCP connections. ansible.builtin.command: mysql --defaults-extra-file={{ database.credentials_file }} -e "CREATE USER IF NOT EXISTS '{{ _mysql_build_user_name }}'@'%' IDENTIFIED BY '{{ _mysql_build_password }}'; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, EVENT, TRIGGER ON `{{ _mysql_build_database_name }}`.* TO '{{ _mysql_build_user_name }}'@'%';" - when: ( mysql_backup.credentials_handling == 'rotate' ) or ( mysql_backup.credentials_handling == 'static' ) + when: + - ( mysql_backup.credentials_handling == 'rotate' ) or ( mysql_backup.credentials_handling == 'static' ) + - database.is_replica is not defined run_once: true - name: Create/update mysql user for unix socket connections. ansible.builtin.command: mysql --defaults-extra-file={{ database.credentials_file }} -e "CREATE USER IF NOT EXISTS '{{ _mysql_build_user_name }}'@'localhost' IDENTIFIED BY '{{ _mysql_build_password }}'; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, EVENT, TRIGGER ON `{{ _mysql_build_database_name }}`.* TO '{{ _mysql_build_user_name }}'@'localhost';" - when: ( mysql_backup.credentials_handling == 'rotate' ) or ( mysql_backup.credentials_handling == 'static' ) + when: + - ( mysql_backup.credentials_handling == 'rotate' ) or ( mysql_backup.credentials_handling == 'static' ) + - database.is_replica is not defined run_once: true - ansible.builtin.set_fact: diff --git a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml index e99a8bda..6040c73a 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert-dump.yml @@ -19,9 +19,7 @@ args: executable: /bin/bash run_once: true - when: - - previous_build_number > 0 - - database.is_replica is not defined + when: previous_build_number > 0 - name: Delete unpacked dump file. ansible.builtin.file: diff --git a/roles/database_backup/database_backup-mysql/tasks/revert.yml b/roles/database_backup/database_backup-mysql/tasks/revert.yml index f6076262..9e698c91 100644 --- a/roles/database_backup/database_backup-mysql/tasks/revert.yml +++ b/roles/database_backup/database_backup-mysql/tasks/revert.yml @@ -17,3 +17,4 @@ when: - previous_build_number > 0 - database_backup.revert + - database.is_replica is not defined From e809b4dbfb0a7683942be4c3352089bc38077876 Mon Sep 17 00:00:00 2001 From: Philip Norton Date: Thu, 4 Apr 2024 15:36:46 +0100 Subject: [PATCH 118/190] 448 muti site d10 mr devel (#450) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton --- .../tasks/settings.yml | 14 + .../templates/default.settings.php.j2 | 883 ++++++++++++++++++ 2 files changed, 897 insertions(+) create mode 100644 roles/config_generate/config_generate-drupal8/templates/default.settings.php.j2 diff --git a/roles/config_generate/config_generate-drupal8/tasks/settings.yml b/roles/config_generate/config_generate-drupal8/tasks/settings.yml index 79e0ad5e..2b07a09b 100644 --- a/roles/config_generate/config_generate-drupal8/tasks/settings.yml +++ b/roles/config_generate/config_generate-drupal8/tasks/settings.yml @@ -31,3 +31,17 @@ - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.{{ config_generate_drupal.drupal_settings_file_name }}" - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ config_generate_drupal.drupal_settings_file_name }}" - "settings.php.j2" + +- name: Generates default.settings.php file in any additional multi-site directories. + ansible.builtin.template: + src: "{{ item }}" + dest: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}/default.settings.php" + with_first_found: + - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/default.settings.php" + - "{{ playbook_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.default.settings.php" + - "{{ deploy_path }}/{{ webroot }}/sites/default/default.settings.php" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/{{ build_type }}.default.{{ config_generate_drupal.drupal_settings_file_name }}" + - "{{ _ce_deploy_build_dir }}/{{ webroot }}/sites/{{ site.folder }}/default.{{ config_generate_drupal.drupal_settings_file_name }}" + - "default.settings.php.j2" + when: + - site.folder != "default" diff --git a/roles/config_generate/config_generate-drupal8/templates/default.settings.php.j2 b/roles/config_generate/config_generate-drupal8/templates/default.settings.php.j2 new file mode 100644 index 00000000..63fb2df7 --- /dev/null +++ b/roles/config_generate/config_generate-drupal8/templates/default.settings.php.j2 @@ -0,0 +1,883 @@ + 'databasename', + * 'username' => 'sql_username', + * 'password' => 'sql_password', + * 'host' => 'localhost', + * 'port' => '3306', + * 'driver' => 'mysql', + * 'prefix' => '', + * 'collation' => 'utf8mb4_general_ci', + * ]; + * @endcode + */ +$databases = []; + +/** + * Customizing database settings. + * + * Many of the values of the $databases array can be customized for your + * particular database system. Refer to the sample in the section above as a + * starting point. + * + * The "driver" property indicates what Drupal database driver the + * connection should use. This is usually the same as the name of the + * database type, such as mysql or sqlite, but not always. The other + * properties will vary depending on the driver. For SQLite, you must + * specify a database file name in a directory that is writable by the + * webserver. For most other drivers, you must specify a + * username, password, host, and database name. + * + * Drupal core implements drivers for mysql, pgsql, and sqlite. Other drivers + * can be provided by contributed or custom modules. To use a contributed or + * custom driver, the "namespace" property must be set to the namespace of the + * driver. The code in this namespace must be autoloadable prior to connecting + * to the database, and therefore, prior to when module root namespaces are + * added to the autoloader. To add the driver's namespace to the autoloader, + * set the "autoload" property to the PSR-4 base directory of the driver's + * namespace. This is optional for projects managed with Composer if the + * driver's namespace is in Composer's autoloader. + * + * For each database, you may optionally specify multiple "target" databases. + * A target database allows Drupal to try to send certain queries to a + * different database if it can but fall back to the default connection if not. + * That is useful for primary/replica replication, as Drupal may try to connect + * to a replica server when appropriate and if one is not available will simply + * fall back to the single primary server (The terms primary/replica are + * traditionally referred to as master/slave in database server documentation). + * + * The general format for the $databases array is as follows: + * @code + * $databases['default']['default'] = $info_array; + * $databases['default']['replica'][] = $info_array; + * $databases['default']['replica'][] = $info_array; + * $databases['extra']['default'] = $info_array; + * @endcode + * + * In the above example, $info_array is an array of settings described above. + * The first line sets a "default" database that has one primary database + * (the second level default). The second and third lines create an array + * of potential replica databases. Drupal will select one at random for a given + * request as needed. The fourth line creates a new database with a name of + * "extra". + * + * For MySQL, MariaDB or equivalent databases the 'isolation_level' option can + * be set. The recommended transaction isolation level for Drupal sites is + * 'READ COMMITTED'. The 'REPEATABLE READ' option is supported but can result + * in deadlocks, the other two options are 'READ UNCOMMITTED' and 'SERIALIZABLE'. + * They are available but not supported; use them at your own risk. For more + * info: + * https://dev.mysql.com/doc/refman/5.7/en/innodb-transaction-isolation-levels.html + * + * On your settings.php, change the isolation level: + * @code + * $databases['default']['default']['init_commands'] = [ + * 'isolation_level' => 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED', + * ]; + * @endcode + * + * You can optionally set a prefix for all database table names by using the + * 'prefix' setting. If a prefix is specified, the table name will be prepended + * with its value. Be sure to use valid database characters only, usually + * alphanumeric and underscore. If no prefix is desired, do not set the 'prefix' + * key or set its value to an empty string ''. + * + * For example, to have all database table prefixed with 'main_', set: + * @code + * 'prefix' => 'main_', + * @endcode + * + * Advanced users can add or override initial commands to execute when + * connecting to the database server, as well as PDO connection settings. For + * example, to enable MySQL SELECT queries to exceed the max_join_size system + * variable, and to reduce the database connection timeout to 5 seconds: + * @code + * $databases['default']['default'] = [ + * 'init_commands' => [ + * 'big_selects' => 'SET SQL_BIG_SELECTS=1', + * ], + * 'pdo' => [ + * PDO::ATTR_TIMEOUT => 5, + * ], + * ]; + * @endcode + * + * WARNING: The above defaults are designed for database portability. Changing + * them may cause unexpected behavior, including potential data loss. See + * https://www.drupal.org/developing/api/database/configuration for more + * information on these defaults and the potential issues. + * + * More details can be found in the constructor methods for each driver: + * - \Drupal\mysql\Driver\Database\mysql\Connection::__construct() + * - \Drupal\pgsql\Driver\Database\pgsql\Connection::__construct() + * - \Drupal\sqlite\Driver\Database\sqlite\Connection::__construct() + * + * Sample Database configuration format for PostgreSQL (pgsql): + * @code + * $databases['default']['default'] = [ + * 'driver' => 'pgsql', + * 'database' => 'databasename', + * 'username' => 'sql_username', + * 'password' => 'sql_password', + * 'host' => 'localhost', + * 'prefix' => '', + * ]; + * @endcode + * + * Sample Database configuration format for SQLite (sqlite): + * @code + * $databases['default']['default'] = [ + * 'driver' => 'sqlite', + * 'database' => '/path/to/database_filename', + * ]; + * @endcode + * + * Sample Database configuration format for a driver in a contributed module: + * @code + * $databases['default']['default'] = [ + * 'driver' => 'my_driver', + * 'namespace' => 'Drupal\my_module\Driver\Database\my_driver', + * 'autoload' => 'modules/my_module/src/Driver/Database/my_driver/', + * 'database' => 'databasename', + * 'username' => 'sql_username', + * 'password' => 'sql_password', + * 'host' => 'localhost', + * 'prefix' => '', + * ]; + * @endcode + * + * Sample Database configuration format for a driver that is extending another + * database driver. + * @code + * $databases['default']['default'] = [ + * 'driver' => 'my_driver', + * 'namespace' => 'Drupal\my_module\Driver\Database\my_driver', + * 'autoload' => 'modules/my_module/src/Driver/Database/my_driver/', + * 'database' => 'databasename', + * 'username' => 'sql_username', + * 'password' => 'sql_password', + * 'host' => 'localhost', + * 'prefix' => '', + * 'dependencies' => [ + * 'parent_module' => [ + * 'namespace' => 'Drupal\parent_module', + * 'autoload' => 'core/modules/parent_module/src/', + * ], + * ], + * ]; + * @endcode + */ + +/** + * Location of the site configuration files. + * + * The $settings['config_sync_directory'] specifies the location of file system + * directory used for syncing configuration data. On install, the directory is + * created. This is used for configuration imports. + * + * The default location for this directory is inside a randomly-named + * directory in the public files path. The setting below allows you to set + * its location. + */ +# $settings['config_sync_directory'] = '/directory/outside/webroot'; + +/** + * Settings: + * + * $settings contains environment-specific configuration, such as the files + * directory and reverse proxy address, and temporary configuration, such as + * security overrides. + * + * @see \Drupal\Core\Site\Settings::get() + */ + +/** + * Salt for one-time login links, cancel links, form tokens, etc. + * + * This variable will be set to a random value by the installer. All one-time + * login links will be invalidated if the value is changed. Note that if your + * site is deployed on a cluster of web servers, you must ensure that this + * variable has the same value on each server. + * + * For enhanced security, you may set this variable to the contents of a file + * outside your document root, and vary the value across environments (like + * production and development); you should also ensure that this file is not + * stored with backups of your database. + * + * Example: + * @code + * $settings['hash_salt'] = file_get_contents('/home/example/salt.txt'); + * @endcode + */ +$settings['hash_salt'] = ''; + +/** + * Deployment identifier. + * + * Drupal's dependency injection container will be automatically invalidated and + * rebuilt when the Drupal core version changes. When updating contributed or + * custom code that changes the container, changing this identifier will also + * allow the container to be invalidated as soon as code is deployed. + */ +# $settings['deployment_identifier'] = \Drupal::VERSION; + +/** + * Access control for update.php script. + * + * If you are updating your Drupal installation using the update.php script but + * are not logged in using either an account with the "Administer software + * updates" permission or the site maintenance account (the account that was + * created during installation), you will need to modify the access check + * statement below. Change the FALSE to a TRUE to disable the access check. + * After finishing the upgrade, be sure to open this file again and change the + * TRUE back to a FALSE! + */ +$settings['update_free_access'] = FALSE; + +/** + * Fallback to HTTP for Update Manager and for fetching security advisories. + * + * If your site fails to connect to updates.drupal.org over HTTPS (either when + * fetching data on available updates, or when fetching the feed of critical + * security announcements), you may uncomment this setting and set it to TRUE to + * allow an insecure fallback to HTTP. Note that doing so will open your site up + * to a potential man-in-the-middle attack. You should instead attempt to + * resolve the issues before enabling this option. + * @see https://www.drupal.org/docs/system-requirements/php-requirements#openssl + * @see https://en.wikipedia.org/wiki/Man-in-the-middle_attack + * @see \Drupal\update\UpdateFetcher + * @see \Drupal\system\SecurityAdvisories\SecurityAdvisoriesFetcher + */ +# $settings['update_fetch_with_http_fallback'] = TRUE; + +/** + * External access proxy settings: + * + * If your site must access the Internet via a web proxy then you can enter the + * proxy settings here. Set the full URL of the proxy, including the port, in + * variables: + * - $settings['http_client_config']['proxy']['http']: The proxy URL for HTTP + * requests. + * - $settings['http_client_config']['proxy']['https']: The proxy URL for HTTPS + * requests. + * You can pass in the user name and password for basic authentication in the + * URLs in these settings. + * + * You can also define an array of host names that can be accessed directly, + * bypassing the proxy, in $settings['http_client_config']['proxy']['no']. + */ +# $settings['http_client_config']['proxy']['http'] = 'http://proxy_user:proxy_pass@example.com:8080'; +# $settings['http_client_config']['proxy']['https'] = 'http://proxy_user:proxy_pass@example.com:8080'; +# $settings['http_client_config']['proxy']['no'] = ['127.0.0.1', 'localhost']; + +/** + * Reverse Proxy Configuration: + * + * Reverse proxy servers are often used to enhance the performance + * of heavily visited sites and may also provide other site caching, + * security, or encryption benefits. In an environment where Drupal + * is behind a reverse proxy, the real IP address of the client should + * be determined such that the correct client IP address is available + * to Drupal's logging, statistics, and access management systems. In + * the most simple scenario, the proxy server will add an + * X-Forwarded-For header to the request that contains the client IP + * address. However, HTTP headers are vulnerable to spoofing, where a + * malicious client could bypass restrictions by setting the + * X-Forwarded-For header directly. Therefore, Drupal's proxy + * configuration requires the IP addresses of all remote proxies to be + * specified in $settings['reverse_proxy_addresses'] to work correctly. + * + * Enable this setting to get Drupal to determine the client IP from the + * X-Forwarded-For header. If you are unsure about this setting, do not have a + * reverse proxy, or Drupal operates in a shared hosting environment, this + * setting should remain commented out. + * + * In order for this setting to be used you must specify every possible + * reverse proxy IP address in $settings['reverse_proxy_addresses']. + * If a complete list of reverse proxies is not available in your + * environment (for example, if you use a CDN) you may set the + * $_SERVER['REMOTE_ADDR'] variable directly in settings.php. + * Be aware, however, that it is likely that this would allow IP + * address spoofing unless more advanced precautions are taken. + */ +# $settings['reverse_proxy'] = TRUE; + +/** + * Reverse proxy addresses. + * + * Specify every reverse proxy IP address in your environment, as an array of + * IPv4/IPv6 addresses or subnets in CIDR notation. This setting is required if + * $settings['reverse_proxy'] is TRUE. + */ +# $settings['reverse_proxy_addresses'] = ['a.b.c.d', 'e.f.g.h/24', ...]; + +/** + * Reverse proxy trusted headers. + * + * Sets which headers to trust from your reverse proxy. + * + * Common values are: + * - \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_FOR + * - \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_HOST + * - \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PORT + * - \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PROTO + * - \Symfony\Component\HttpFoundation\Request::HEADER_FORWARDED + * + * Note the default value of + * @code + * \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_FOR | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_HOST | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PORT | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PROTO | \Symfony\Component\HttpFoundation\Request::HEADER_FORWARDED + * @endcode + * is not secure by default. The value should be set to only the specific + * headers the reverse proxy uses. For example: + * @code + * \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_FOR | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_HOST | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PORT | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PROTO + * @endcode + * This would trust the following headers: + * - X_FORWARDED_FOR + * - X_FORWARDED_HOST + * - X_FORWARDED_PROTO + * - X_FORWARDED_PORT + * + * @see \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_FOR + * @see \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_HOST + * @see \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PORT + * @see \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PROTO + * @see \Symfony\Component\HttpFoundation\Request::HEADER_FORWARDED + * @see \Symfony\Component\HttpFoundation\Request::setTrustedProxies + */ +# $settings['reverse_proxy_trusted_headers'] = \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_FOR | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_HOST | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PORT | \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_PROTO | \Symfony\Component\HttpFoundation\Request::HEADER_FORWARDED; + + +/** + * Page caching: + * + * By default, Drupal sends a "Vary: Cookie" HTTP header for anonymous page + * views. This tells a HTTP proxy that it may return a page from its local + * cache without contacting the web server, if the user sends the same Cookie + * header as the user who originally requested the cached page. Without "Vary: + * Cookie", authenticated users would also be served the anonymous page from + * the cache. If the site has mostly anonymous users except a few known + * editors/administrators, the Vary header can be omitted. This allows for + * better caching in HTTP proxies (including reverse proxies), i.e. even if + * clients send different cookies, they still get content served from the cache. + * However, authenticated users should access the site directly (i.e. not use an + * HTTP proxy, and bypass the reverse proxy if one is used) in order to avoid + * getting cached pages from the proxy. + */ +# $settings['omit_vary_cookie'] = TRUE; + + +/** + * Cache TTL for client error (4xx) responses. + * + * Items cached per-URL tend to result in a large number of cache items, and + * this can be problematic on 404 pages which by their nature are unbounded. A + * fixed TTL can be set for these items, defaulting to one hour, so that cache + * backends which do not support LRU can purge older entries. To disable caching + * of client error responses set the value to 0. Currently applies only to + * page_cache module. + */ +# $settings['cache_ttl_4xx'] = 3600; + +/** + * Expiration of cached forms. + * + * Drupal's Form API stores details of forms in a cache and these entries are + * kept for at least 6 hours by default. Expired entries are cleared by cron. + * + * @see \Drupal\Core\Form\FormCache::setCache() + */ +# $settings['form_cache_expiration'] = 21600; + +/** + * Class Loader. + * + * If the APCu extension is detected, the classloader will be optimized to use + * it. Set to FALSE to disable this. + * + * @see https://getcomposer.org/doc/articles/autoloader-optimization.md + */ +# $settings['class_loader_auto_detect'] = FALSE; + +/** + * Authorized file system operations: + * + * The Update Manager module included with Drupal provides a mechanism for + * site administrators to securely install missing updates for the site + * directly through the web user interface. On securely-configured servers, + * the Update manager will require the administrator to provide SSH or FTP + * credentials before allowing the installation to proceed; this allows the + * site to update the new files as the user who owns all the Drupal files, + * instead of as the user the webserver is running as. On servers where the + * webserver user is itself the owner of the Drupal files, the administrator + * will not be prompted for SSH or FTP credentials (note that these server + * setups are common on shared hosting, but are inherently insecure). + * + * Some sites might wish to disable the above functionality, and only update + * the code directly via SSH or FTP themselves. This setting completely + * disables all functionality related to these authorized file operations. + * + * @see https://www.drupal.org/node/244924 + * + * Remove the leading hash signs to disable. + */ +# $settings['allow_authorize_operations'] = FALSE; + +/** + * Default mode for directories and files written by Drupal. + * + * Value should be in PHP Octal Notation, with leading zero. + */ +# $settings['file_chmod_directory'] = 0775; +# $settings['file_chmod_file'] = 0664; + +/** + * Optimized assets path: + * + * A local file system path where optimized assets will be stored. This directory + * must exist and be writable by Drupal. This directory must be relative to + * the Drupal installation directory and be accessible over the web. + */ +# $settings['file_assets_path'] = 'sites/default/files'; + +/** + * Public file base URL: + * + * An alternative base URL to be used for serving public files. This must + * include any leading directory path. + * + * A different value from the domain used by Drupal to be used for accessing + * public files. This can be used for a simple CDN integration, or to improve + * security by serving user-uploaded files from a different domain or subdomain + * pointing to the same server. Do not include a trailing slash. + */ +# $settings['file_public_base_url'] = 'http://downloads.example.com/files'; + +/** + * Public file path: + * + * A local file system path where public files will be stored. This directory + * must exist and be writable by Drupal. This directory must be relative to + * the Drupal installation directory and be accessible over the web. + */ +# $settings['file_public_path'] = 'sites/default/files'; + +/** + * Additional public file schemes: + * + * Public schemes are URI schemes that allow download access to all users for + * all files within that scheme. + * + * The "public" scheme is always public, and the "private" scheme is always + * private, but other schemes, such as "https", "s3", "example", or others, + * can be either public or private depending on the site. By default, they're + * private, and access to individual files is controlled via + * hook_file_download(). + * + * Typically, if a scheme should be public, a module makes it public by + * implementing hook_file_download(), and granting access to all users for all + * files. This could be either the same module that provides the stream wrapper + * for the scheme, or a different module that decides to make the scheme + * public. However, in cases where a site needs to make a scheme public, but + * is unable to add code in a module to do so, the scheme may be added to this + * variable, the result of which is that system_file_download() grants public + * access to all files within that scheme. + */ +# $settings['file_additional_public_schemes'] = ['example']; + +/** + * File schemes whose paths should not be normalized: + * + * Normally, Drupal normalizes '/./' and '/../' segments in file URIs in order + * to prevent unintended file access. For example, 'private://css/../image.png' + * is normalized to 'private://image.png' before checking access to the file. + * + * On Windows, Drupal also replaces '\' with '/' in URIs for the local + * filesystem. + * + * If file URIs with one or more scheme should not be normalized like this, then + * list the schemes here. For example, if 'porcelain://china/./plate.png' should + * not be normalized to 'porcelain://china/plate.png', then add 'porcelain' to + * this array. In this case, make sure that the module providing the 'porcelain' + * scheme does not allow unintended file access when using '/../' to move up the + * directory tree. + */ +# $settings['file_sa_core_2023_005_schemes'] = ['porcelain']; + +/** + * Configuration for phpinfo() admin status report. + * + * Drupal's admin UI includes a report at admin/reports/status/php which shows + * the output of phpinfo(). The full output can contain sensitive information + * so by default Drupal removes some sections. + * + * This behavior can be configured by setting this variable to a different + * value corresponding to the flags parameter of phpinfo(). + * + * If you need to expose more information in the report - for example to debug a + * problem - consider doing so temporarily. + * + * @see https://www.php.net/manual/function.phpinfo.php + */ +# $settings['sa_core_2023_004_phpinfo_flags'] = ~ (INFO_VARIABLES | INFO_ENVIRONMENT); + +/** + * Private file path: + * + * A local file system path where private files will be stored. This directory + * must be absolute, outside of the Drupal installation directory and not + * accessible over the web. + * + * Note: Caches need to be cleared when this value is changed to make the + * private:// stream wrapper available to the system. + * + * See https://www.drupal.org/documentation/modules/file for more information + * about securing private files. + */ +# $settings['file_private_path'] = ''; + +/** + * Temporary file path: + * + * A local file system path where temporary files will be stored. This directory + * must be absolute, outside of the Drupal installation directory and not + * accessible over the web. + * + * If this is not set, the default for the operating system will be used. + * + * @see \Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory() + */ +# $settings['file_temp_path'] = '/tmp'; + +/** + * Session write interval: + * + * Set the minimum interval between each session write to database. + * For performance reasons it defaults to 180. + */ +# $settings['session_write_interval'] = 180; + +/** + * String overrides: + * + * To override specific strings on your site with or without enabling the Locale + * module, add an entry to this list. This functionality allows you to change + * a small number of your site's default English language interface strings. + * + * Remove the leading hash signs to enable. + * + * The "en" part of the variable name, is dynamic and can be any langcode of + * any added language. (eg locale_custom_strings_de for german). + */ +# $settings['locale_custom_strings_en'][''] = [ +# 'Home' => 'Front page', +# '@count min' => '@count minutes', +# ]; + +/** + * A custom theme for the offline page: + * + * This applies when the site is explicitly set to maintenance mode through the + * administration page or when the database is inactive due to an error. + * The template file should also be copied into the theme. It is located inside + * 'core/modules/system/templates/maintenance-page.html.twig'. + * + * Note: This setting does not apply to installation and update pages. + */ +# $settings['maintenance_theme'] = 'claro'; + +/** + * PHP settings: + * + * To see what PHP settings are possible, including whether they can be set at + * runtime (by using ini_set()), read the PHP documentation: + * http://php.net/manual/ini.list.php + * See \Drupal\Core\DrupalKernel::bootEnvironment() for required runtime + * settings and the .htaccess file for non-runtime settings. + * Settings defined there should not be duplicated here so as to avoid conflict + * issues. + */ + +/** + * If you encounter a situation where users post a large amount of text, and + * the result is stripped out upon viewing but can still be edited, Drupal's + * output filter may not have sufficient memory to process it. If you + * experience this issue, you may wish to uncomment the following two lines + * and increase the limits of these variables. For more information, see + * http://php.net/manual/pcre.configuration.php. + */ +# ini_set('pcre.backtrack_limit', 200000); +# ini_set('pcre.recursion_limit', 200000); + +/** + * Configuration overrides. + * + * To globally override specific configuration values for this site, + * set them here. You usually don't need to use this feature. This is + * useful in a configuration file for a vhost or directory, rather than + * the default settings.php. + * + * Note that any values you provide in these variable overrides will not be + * viewable from the Drupal administration interface. The administration + * interface displays the values stored in configuration so that you can stage + * changes to other environments that don't have the overrides. + * + * There are particular configuration values that are risky to override. For + * example, overriding the list of installed modules in 'core.extension' is not + * supported as module install or uninstall has not occurred. Other examples + * include field storage configuration, because it has effects on database + * structure, and 'core.menu.static_menu_link_overrides' since this is cached in + * a way that is not config override aware. Also, note that changing + * configuration values in settings.php will not fire any of the configuration + * change events. + */ +# $config['system.site']['name'] = 'My Drupal site'; +# $config['user.settings']['anonymous'] = 'Visitor'; + +/** + * Load services definition file. + */ +$settings['container_yamls'][] = $app_root . '/' . $site_path . '/services.yml'; + +/** + * Override the default service container class. + * + * This is useful for example to trace the service container for performance + * tracking purposes, for testing a service container with an error condition or + * to test a service container that throws an exception. + */ +# $settings['container_base_class'] = '\Drupal\Core\DependencyInjection\Container'; + +/** + * Override the default yaml parser class. + * + * Provide a fully qualified class name here if you would like to provide an + * alternate implementation YAML parser. The class must implement the + * \Drupal\Component\Serialization\SerializationInterface interface. + */ +# $settings['yaml_parser_class'] = NULL; + +/** + * Trusted host configuration. + * + * Drupal core can use the Symfony trusted host mechanism to prevent HTTP Host + * header spoofing. + * + * To enable the trusted host mechanism, you enable your allowable hosts + * in $settings['trusted_host_patterns']. This should be an array of regular + * expression patterns, without delimiters, representing the hosts you would + * like to allow. + * + * For example: + * @code + * $settings['trusted_host_patterns'] = [ + * '^www\.example\.com$', + * ]; + * @endcode + * will allow the site to only run from www.example.com. + * + * If you are running multisite, or if you are running your site from + * different domain names (eg, you don't redirect http://www.example.com to + * http://example.com), you should specify all of the host patterns that are + * allowed by your site. + * + * For example: + * @code + * $settings['trusted_host_patterns'] = [ + * '^example\.com$', + * '^.+\.example\.com$', + * '^example\.org$', + * '^.+\.example\.org$', + * ]; + * @endcode + * will allow the site to run off of all variants of example.com and + * example.org, with all subdomains included. + * + * @see https://www.drupal.org/docs/installing-drupal/trusted-host-settings + */ +# $settings['trusted_host_patterns'] = []; + +/** + * The default list of directories that will be ignored by Drupal's file API. + * + * By default ignore node_modules and bower_components folders to avoid issues + * with common frontend tools and recursive scanning of directories looking for + * extensions. + * + * @see \Drupal\Core\File\FileSystemInterface::scanDirectory() + * @see \Drupal\Core\Extension\ExtensionDiscovery::scanDirectory() + */ +$settings['file_scan_ignore_directories'] = [ + 'node_modules', + 'bower_components', +]; + +/** + * The default number of entities to update in a batch process. + * + * This is used by update and post-update functions that need to go through and + * change all the entities on a site, so it is useful to increase this number + * if your hosting configuration (i.e. RAM allocation, CPU speed) allows for a + * larger number of entities to be processed in a single batch run. + */ +$settings['entity_update_batch_size'] = 50; + +/** + * Entity update backup. + * + * This is used to inform the entity storage handler that the backup tables as + * well as the original entity type and field storage definitions should be + * retained after a successful entity update process. + */ +$settings['entity_update_backup'] = TRUE; + +/** + * Node migration type. + * + * This is used to force the migration system to use the classic node migrations + * instead of the default complete node migrations. The migration system will + * use the classic node migration only if there are existing migrate_map tables + * for the classic node migrations and they contain data. These tables may not + * exist if you are developing custom migrations and do not want to use the + * complete node migrations. Set this to TRUE to force the use of the classic + * node migrations. + */ +$settings['migrate_node_migrate_type_classic'] = FALSE; + +/** + * The default settings for migration sources. + * + * These settings are used as the default settings on the Credential form at + * /upgrade/credentials. + * + * - migrate_source_version - The version of the source database. This can be + * '6' or '7'. Defaults to '7'. + * - migrate_source_connection - The key in the $databases array for the source + * site. + * - migrate_file_public_path - The location of the source Drupal 6 or Drupal 7 + * public files. This can be a local file directory containing the source + * Drupal 6 or Drupal 7 site (e.g /var/www/docroot), or the site address + * (e.g http://example.com). + * - migrate_file_private_path - The location of the source Drupal 7 private + * files. This can be a local file directory containing the source Drupal 7 + * site (e.g /var/www/docroot), or empty to use the same value as Public + * files directory. + * + * Sample configuration for a drupal 6 source site with the source files in a + * local directory. + * + * @code + * $settings['migrate_source_version'] = '6'; + * $settings['migrate_source_connection'] = 'migrate'; + * $settings['migrate_file_public_path'] = '/var/www/drupal6'; + * @endcode + * + * Sample configuration for a drupal 7 source site with public source files on + * the source site and the private files in a local directory. + * + * @code + * $settings['migrate_source_version'] = '7'; + * $settings['migrate_source_connection'] = 'migrate'; + * $settings['migrate_file_public_path'] = 'https://drupal7.com'; + * $settings['migrate_file_private_path'] = '/var/www/drupal7'; + * @endcode + */ +# $settings['migrate_source_connection'] = ''; +# $settings['migrate_source_version'] = ''; +# $settings['migrate_file_public_path'] = ''; +# $settings['migrate_file_private_path'] = ''; + +/** + * Load local development override configuration, if available. + * + * Create a settings.local.php file to override variables on secondary (staging, + * development, etc.) installations of this site. + * + * Typical uses of settings.local.php include: + * - Disabling caching. + * - Disabling JavaScript/CSS compression. + * - Rerouting outgoing emails. + * + * Keep this code block at the end of this file to take full effect. + */ +# +# if (file_exists($app_root . '/' . $site_path . '/settings.local.php')) { +# include $app_root . '/' . $site_path . '/settings.local.php'; +# } From edbf13e0fca714b7c5922fa47d06988615d9983d Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 4 Apr 2024 16:38:20 +0200 Subject: [PATCH 119/190] Catching up devel (#452) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton From 2514962aefe15bf05e68fd99e593ad8653703ba5 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 4 Apr 2024 18:31:10 +0200 Subject: [PATCH 120/190] Statuscake maintenance role pr devel (#453) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- docs/_Sidebar.md | 1 + docs/roles/maintenance_mode.md | 26 +++++++++++++++++++ roles/maintenance_mode/README.md | 26 +++++++++++++++++++ roles/maintenance_mode/defaults/main.yml | 10 +++---- .../maintenance_mode-nginx/defaults/main.yml | 2 +- .../defaults/main.yml | 5 ++++ .../tasks/main.yml | 8 ++++++ .../tasks/offline.yml | 17 ++++++++++++ .../tasks/online.yml | 17 ++++++++++++ 9 files changed, 106 insertions(+), 6 deletions(-) create mode 100644 docs/roles/maintenance_mode.md create mode 100644 roles/maintenance_mode/README.md create mode 100644 roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml create mode 100644 roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml create mode 100644 roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml create mode 100644 roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index 3d47469a..a58673b3 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -34,6 +34,7 @@ - [Deploy container](/roles/deploy_container) - [Init](/roles/_init) - [LHCI run](/roles/lhci_run) + - [Maintenance Mode](/roles/maintenance_mode) - ["Meta"](/roles/_meta) - [Drupal 7](/roles/_meta/deploy-drupal7) - [Drupal 8](/roles/_meta/deploy-drupal8) diff --git a/docs/roles/maintenance_mode.md b/docs/roles/maintenance_mode.md new file mode 100644 index 00000000..ec9d640c --- /dev/null +++ b/docs/roles/maintenance_mode.md @@ -0,0 +1,26 @@ +# Maintenance Mode +This role and its sub-roles handle various methods for putting applications into an offline maintenance state. See each sub-role's own defaults for additional variables. + + + + + +## Default variables +```yaml +--- +# Puts site(s) offline. +maintenance_mode: + # Where do we apply maintenance - can be invoked more than one time for different purposes. + # - nginx: serves a static maintenance page + # - drupal-core: application level + # - statuscake: disables a StatusCake check + # @todo - haproxy: serves a static maintenance page. + # @todo - drupal-read-only: application level via readonly module. + mode: "nginx" + # - offline: puts the site offline + # @todo - restricted: put the site offline except for whitelist (nginx/haproxy only) + # - online: brings the site back online. + operation: "offline" +``` + + diff --git a/roles/maintenance_mode/README.md b/roles/maintenance_mode/README.md new file mode 100644 index 00000000..ec9d640c --- /dev/null +++ b/roles/maintenance_mode/README.md @@ -0,0 +1,26 @@ +# Maintenance Mode +This role and its sub-roles handle various methods for putting applications into an offline maintenance state. See each sub-role's own defaults for additional variables. + + + + + +## Default variables +```yaml +--- +# Puts site(s) offline. +maintenance_mode: + # Where do we apply maintenance - can be invoked more than one time for different purposes. + # - nginx: serves a static maintenance page + # - drupal-core: application level + # - statuscake: disables a StatusCake check + # @todo - haproxy: serves a static maintenance page. + # @todo - drupal-read-only: application level via readonly module. + mode: "nginx" + # - offline: puts the site offline + # @todo - restricted: put the site offline except for whitelist (nginx/haproxy only) + # - online: brings the site back online. + operation: "offline" +``` + + diff --git a/roles/maintenance_mode/defaults/main.yml b/roles/maintenance_mode/defaults/main.yml index ab895227..15c050e9 100644 --- a/roles/maintenance_mode/defaults/main.yml +++ b/roles/maintenance_mode/defaults/main.yml @@ -1,12 +1,12 @@ --- # Puts site(s) offline. maintenance_mode: - # What level do we operate. - # - nginx: serves a static maintenance page. - # - drupal_core: application level - # - mautic: application level + # Where do we apply maintenance - can be invoked more than one time for different purposes. + # - nginx: serves a static maintenance page + # - drupal-core: application level + # - statuscake: disables a StatusCake check # @todo - haproxy: serves a static maintenance page. - # @todo - drupal_read_only: application level via readonly module. + # @todo - drupal-read-only: application level via readonly module. mode: "nginx" # - offline: puts the site offline # @todo - restricted: put the site offline except for whitelist (nginx/haproxy only) diff --git a/roles/maintenance_mode/maintenance_mode-nginx/defaults/main.yml b/roles/maintenance_mode/maintenance_mode-nginx/defaults/main.yml index 94e4737a..193e6a4d 100644 --- a/roles/maintenance_mode/maintenance_mode-nginx/defaults/main.yml +++ b/roles/maintenance_mode/maintenance_mode-nginx/defaults/main.yml @@ -2,4 +2,4 @@ maintenance_mode_nginx: domains: - server_name: "{{ domain_name }}" - maintenance_page: "index.html" \ No newline at end of file + maintenance_page: "index.html" diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml b/roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml new file mode 100644 index 00000000..097a55bd --- /dev/null +++ b/roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml @@ -0,0 +1,5 @@ +--- +maintenance_mode_statuscake: + domains: + - statuscake_check_id: "" # Your StatusCake Test ID + statuscake_api_key: "abcdef123456" # Your StatusCake API key - create one on your user page - https://app.statuscake.com/User.php diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml new file mode 100644 index 00000000..15b5cac8 --- /dev/null +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml @@ -0,0 +1,8 @@ +--- +- ansible.builtin.include_tasks: "{{ maintenance_mode.operation }}.yml" + with_items: "{{ maintenance_mode_statuscake.domains }}" + loop_control: + loop_var: domain + run_once: true + when: + - maintenance_mode_statuscake.domains | length > 0 \ No newline at end of file diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml new file mode 100644 index 00000000..3cd5a186 --- /dev/null +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml @@ -0,0 +1,17 @@ +--- +- name: Pause StatusCake check. + ansible.builtin.uri: + url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id }}" + method: PUT + body: + paused: true + headers: + Authorization: "Bearer {{ domain.statuscake_api_key }}" + status_code: + - 200 + - 204 + when: + - domain.statuscake_check_id is defined + - domain.statuscake_check_id | length > 0 + - domain.statuscake_api_key is defined + - domain.statuscake_api_key | length > 0 diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml new file mode 100644 index 00000000..63663630 --- /dev/null +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml @@ -0,0 +1,17 @@ +--- +- name: Resume StatusCake check. + ansible.builtin.uri: + url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id }}" + method: PUT + body: + paused: false + headers: + Authorization: "Bearer {{ domain.statuscake_api_key }}" + status_code: + - 200 + - 204 + when: + - domain.statuscake_check_id is defined + - domain.statuscake_check_id | length > 0 + - domain.statuscake_api_key is defined + - domain.statuscake_api_key | length > 0 From bf29d28b35bc494b9611cf03d5c5c6cd0c57dcb6 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 4 Apr 2024 19:14:31 +0200 Subject: [PATCH 121/190] Statuscake maintenance role pr devel (#455) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- .../maintenance_mode-statuscake/defaults/main.yml | 2 +- .../maintenance_mode-statuscake/tasks/offline.yml | 1 + .../maintenance_mode-statuscake/tasks/online.yml | 1 + 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml b/roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml index 097a55bd..57240d01 100644 --- a/roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml +++ b/roles/maintenance_mode/maintenance_mode-statuscake/defaults/main.yml @@ -1,5 +1,5 @@ --- maintenance_mode_statuscake: domains: - - statuscake_check_id: "" # Your StatusCake Test ID + - statuscake_check_id: "" # Your StatusCake Test ID - must always be surrounded by double-quotes statuscake_api_key: "abcdef123456" # Your StatusCake API key - create one on your user page - https://app.statuscake.com/User.php diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml index 3cd5a186..62081308 100644 --- a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml @@ -3,6 +3,7 @@ ansible.builtin.uri: url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id }}" method: PUT + body_format: form-urlencoded body: paused: true headers: diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml index 63663630..5818ed33 100644 --- a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml @@ -3,6 +3,7 @@ ansible.builtin.uri: url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id }}" method: PUT + body_format: form-urlencoded body: paused: false headers: From b4d20141fea4815620f5c4be1d4c285ad9353534 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 5 Apr 2024 15:57:11 +0200 Subject: [PATCH 122/190] Statuscake maintenance role pr devel (#457) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- roles/_init/tasks/main.yml | 4 +++ .../tasks/online.yml | 29 +++++++++++++++++-- .../maintenance_mode-nginx/tasks/main.yml | 4 +-- .../maintenance_mode-nginx/tasks/offline.yml | 4 +-- .../maintenance_mode-nginx/tasks/online.yml | 4 +-- .../tasks/main.yml | 2 +- .../tasks/offline.yml | 2 +- .../tasks/online.yml | 2 +- roles/maintenance_mode/tasks/main.yml | 2 +- 9 files changed, 41 insertions(+), 12 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index f57be10e..3b04b4ec 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -53,6 +53,10 @@ ansible.builtin.set_fact: live_symlink_build_dest: "{{ live_symlink_build_dest | default('{{ build_base_path }}/live.{{ project_name }}_{{ build_type }}') }}" + - name: Stash current deploy path for later use where we need to manipulate the live site. + ansible.builtin.set_fact: + initial_deploy_path: "{{ deploy_path }}" + - name: Overwrite deploy and live_symlink paths if SquashFS deploy. ansible.builtin.set_fact: deploy_path: "{{ build_path | default('{{ build_path_prefix }}{{ build_number }}') }}" diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml index 6b127f89..ab548596 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml @@ -7,7 +7,20 @@ become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: - - project_type == 'drupal8' + - project_type == "drupal8" + +# We want to also bring up the 'current' site with squashfs builds +- name: Disable live site maintenance mode if this is a squashfs build. + ansible.builtin.command: + cmd: "{{ drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ initial_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + args: + chdir: "{{ initial_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + become: "{{ 'no' if www_user == deploy_user else 'yes' }}" + become_user: "{{ www_user }}" + when: + - project_type == "drupal8" + - deploy_code.mount_type is defined + - deploy_code.mount_type == "squashfs" # For some reason D7 drush doesn't respect 'chdir' with command, using shell instead. - name: Disable maintenance mode D7. @@ -17,4 +30,16 @@ become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: - - project_type == 'drupal7' + - project_type == "drupal7" + +# We want to also bring up the 'current' site with squashfs builds +- name: Disable maintenance mode D7 if this is a squashfs build. + ansible.builtin.shell: + cmd: "{{ drush_bin }} -l {{ site.folder }} vset maintenance_mode 0" + chdir: "{{ initial_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + become: "{{ 'no' if www_user == deploy_user else 'yes' }}" + become_user: "{{ www_user }}" + when: + - project_type == "drupal7" + - deploy_code.mount_type is defined + - deploy_code.mount_type == "squashfs" diff --git a/roles/maintenance_mode/maintenance_mode-nginx/tasks/main.yml b/roles/maintenance_mode/maintenance_mode-nginx/tasks/main.yml index 65531755..5f5377da 100644 --- a/roles/maintenance_mode/maintenance_mode-nginx/tasks/main.yml +++ b/roles/maintenance_mode/maintenance_mode-nginx/tasks/main.yml @@ -1,8 +1,8 @@ --- -- include_tasks: "{{ maintenance_mode.operation }}.yml" +- ansible.builtin.include_tasks: "{{ maintenance_mode.operation }}.yml" - name: Reload Nginx. - service: + ansible.builtin.service: name: nginx state: reloaded become: true diff --git a/roles/maintenance_mode/maintenance_mode-nginx/tasks/offline.yml b/roles/maintenance_mode/maintenance_mode-nginx/tasks/offline.yml index abcd2246..78fe1df5 100644 --- a/roles/maintenance_mode/maintenance_mode-nginx/tasks/offline.yml +++ b/roles/maintenance_mode/maintenance_mode-nginx/tasks/offline.yml @@ -1,6 +1,6 @@ --- - name: Disable active vhost. - file: + ansible.builtin.file: path: "/etc/nginx/sites-enabled/{{ domain.server_name }}.conf" state: absent with_items: "{{ maintenance_mode_nginx.domains }}" @@ -9,7 +9,7 @@ become: true - name: Enable maintenance vhost. - file: + ansible.builtin.file: src: "/etc/nginx/sites-available/{{ domain.server_name }}-maintenance.conf" dest: "/etc/nginx/sites-enabled/{{ domain.server_name }}-maintenance.conf" state: link diff --git a/roles/maintenance_mode/maintenance_mode-nginx/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-nginx/tasks/online.yml index 59d59c5d..7ad26114 100644 --- a/roles/maintenance_mode/maintenance_mode-nginx/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-nginx/tasks/online.yml @@ -1,6 +1,6 @@ --- - name: Disable maintenance vhost. - file: + ansible.builtin.file: path: "/etc/nginx/sites-enabled/{{ domain.server_name }}-maintenance.conf" state: absent with_items: "{{ maintenance_mode_nginx.domains }}" @@ -9,7 +9,7 @@ become: true - name: Enable active vhost. - file: + ansible.builtin.file: src: "/etc/nginx/sites-available/{{ domain.server_name }}.conf" dest: "/etc/nginx/sites-enabled/{{ domain.server_name }}.conf" state: link diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml index 15b5cac8..ea8de053 100644 --- a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/main.yml @@ -5,4 +5,4 @@ loop_var: domain run_once: true when: - - maintenance_mode_statuscake.domains | length > 0 \ No newline at end of file + - maintenance_mode_statuscake.domains | length > 0 diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml index 62081308..1aeb574f 100644 --- a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/offline.yml @@ -1,7 +1,7 @@ --- - name: Pause StatusCake check. ansible.builtin.uri: - url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id }}" + url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id | string }}" method: PUT body_format: form-urlencoded body: diff --git a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml index 5818ed33..1bed8ec3 100644 --- a/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-statuscake/tasks/online.yml @@ -1,7 +1,7 @@ --- - name: Resume StatusCake check. ansible.builtin.uri: - url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id }}" + url: "https://api.statuscake.com/v1/uptime/{{ domain.statuscake_check_id | string }}" method: PUT body_format: form-urlencoded body: diff --git a/roles/maintenance_mode/tasks/main.yml b/roles/maintenance_mode/tasks/main.yml index 2fbf77ee..ed0ef7f0 100644 --- a/roles/maintenance_mode/tasks/main.yml +++ b/roles/maintenance_mode/tasks/main.yml @@ -1,5 +1,5 @@ --- -- include_role: +- ansible.builtin.include_role: name: "maintenance_mode/maintenance_mode-{{ maintenance_mode.mode }}" when: - deploy_operation != 'cleanup' From 7a7ec23b405e2bc7c87f0b9c9d27730cd694dc97 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 5 Apr 2024 16:05:33 +0200 Subject: [PATCH 123/190] Statuscake maintenance role pr devel (#459) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- roles/_init/tasks/main.yml | 4 ++-- .../maintenance_mode-drupal-core/tasks/online.yml | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 3b04b4ec..9f109361 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -53,9 +53,9 @@ ansible.builtin.set_fact: live_symlink_build_dest: "{{ live_symlink_build_dest | default('{{ build_base_path }}/live.{{ project_name }}_{{ build_type }}') }}" - - name: Stash current deploy path for later use where we need to manipulate the live site. + - name: Set the previous deploy's path for later use where we need to manipulate the live site. ansible.builtin.set_fact: - initial_deploy_path: "{{ deploy_path }}" + previous_deploy_path: "{{ initial_deploy_path | default('{{ deploy_path_prefix }}{{ previous_build_number }}') }}" - name: Overwrite deploy and live_symlink paths if SquashFS deploy. ansible.builtin.set_fact: diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml index ab548596..0f420c8d 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml @@ -12,9 +12,9 @@ # We want to also bring up the 'current' site with squashfs builds - name: Disable live site maintenance mode if this is a squashfs build. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ initial_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" args: - chdir: "{{ initial_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + chdir: "{{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: @@ -36,7 +36,7 @@ - name: Disable maintenance mode D7 if this is a squashfs build. ansible.builtin.shell: cmd: "{{ drush_bin }} -l {{ site.folder }} vset maintenance_mode 0" - chdir: "{{ initial_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + chdir: "{{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: From e39cbcebfd0d99c6c1d8d6628f19d4f3f050eb4a Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 5 Apr 2024 16:14:46 +0200 Subject: [PATCH 124/190] Statuscake maintenance role pr devel (#460) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- roles/_init/tasks/main.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 9f109361..68eef579 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -16,6 +16,11 @@ ansible.builtin.set_fact: webroot: "{{ webroot | default('web') }}" +# This is passed from caller. +- name: Gather last known good build number. + ansible.builtin.set_fact: + previous_build_number: "{{ previous_known_build_number }}" + - name: Define build deploy path prefix. ansible.builtin.set_fact: deploy_path_prefix: "{{ deploy_base_path }}/{{ project_name }}_{{ build_type }}_build_" @@ -93,11 +98,6 @@ # register: last_build # when: last_build_symlink.stat.exists -# This is passed from caller. -- name: Gather last known good build number. - ansible.builtin.set_fact: - previous_build_number: "{{ previous_known_build_number }}" - # - set_fact: # previous_build_number: "{{ last_build.stdout }}" # when: From c4b6b8aec0a2f1255c534a5530e273e5e3cb492f Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 5 Apr 2024 17:05:24 +0200 Subject: [PATCH 125/190] Statuscake maintenance role pr devel (#461) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- roles/_init/tasks/drupal8.yml | 4 ++++ roles/_init/tasks/main.yml | 2 +- .../maintenance_mode-drupal-core/tasks/online.yml | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/roles/_init/tasks/drupal8.yml b/roles/_init/tasks/drupal8.yml index 7c64719e..eb3bdae9 100644 --- a/roles/_init/tasks/drupal8.yml +++ b/roles/_init/tasks/drupal8.yml @@ -3,6 +3,10 @@ set_fact: drush_bin: "{{ drush_bin | default('{{ deploy_path }}/vendor/bin/drush') }}" +- name: Define path to drush for previous build. + set_fact: + previous_drush_bin: "{{ previous_drush_bin | default('{{ previous_deploy_path }}/vendor/bin/drush') }}" + - name: Define path to drush for currently live build. set_fact: drush_live_bin: "{{ drush_live_bin | default('{{ live_symlink_dest }}/vendor/bin/drush') }}" diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 68eef579..75523e18 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -60,7 +60,7 @@ - name: Set the previous deploy's path for later use where we need to manipulate the live site. ansible.builtin.set_fact: - previous_deploy_path: "{{ initial_deploy_path | default('{{ deploy_path_prefix }}{{ previous_build_number }}') }}" + previous_deploy_path: "{{ previous_deploy_path | default('{{ deploy_path_prefix }}{{ previous_build_number }}') }}" - name: Overwrite deploy and live_symlink paths if SquashFS deploy. ansible.builtin.set_fact: diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml index 0f420c8d..47a925f6 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml @@ -12,7 +12,7 @@ # We want to also bring up the 'current' site with squashfs builds - name: Disable live site maintenance mode if this is a squashfs build. ansible.builtin.command: - cmd: "{{ drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" + cmd: "{{ previous_drush_bin }} -l {{ site.folder }} state:set system.maintenance_mode 0 --input-format=integer --root {{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" args: chdir: "{{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }}" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" From 2640f03a635928fd902205329e014f0c8977f709 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 26 Apr 2024 15:02:24 +0200 Subject: [PATCH 126/190] Ecs healthcheck vars pr devel (#462) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- docs/roles/deploy_container.md | 4 ++++ roles/deploy_container/README.md | 4 ++++ roles/deploy_container/defaults/main.yml | 4 ++++ roles/deploy_container/tasks/action-create.yml | 7 +++++-- 4 files changed, 17 insertions(+), 2 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index 06847722..ed42996d 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -124,6 +124,10 @@ deploy_container: protocol: http path: / response_codes: "200" + # optional additional healthcheck settings + #interval: 60 + #healthy_threshold_count: 3 + #unhealthy_threshold_count: 5 # Requires the deploy IAM user to have the managed AWSCertificateManagerFullAccess and AmazonRoute53FullAccess policies attached acm: # see https://github.com/codeenigma/ce-provision/tree/1.x/roles/aws/aws_acm create_cert: false diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index 06847722..ed42996d 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -124,6 +124,10 @@ deploy_container: protocol: http path: / response_codes: "200" + # optional additional healthcheck settings + #interval: 60 + #healthy_threshold_count: 3 + #unhealthy_threshold_count: 5 # Requires the deploy IAM user to have the managed AWSCertificateManagerFullAccess and AmazonRoute53FullAccess policies attached acm: # see https://github.com/codeenigma/ce-provision/tree/1.x/roles/aws/aws_acm create_cert: false diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index ff99d2b9..98b23723 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -81,6 +81,10 @@ deploy_container: protocol: http path: / response_codes: "200" + # optional additional healthcheck settings + #interval: 60 + #healthy_threshold_count: 3 + #unhealthy_threshold_count: 5 # Requires the deploy IAM user to have the managed AWSCertificateManagerFullAccess and AmazonRoute53FullAccess policies attached acm: # see https://github.com/codeenigma/ce-provision/tree/1.x/roles/aws/aws_acm create_cert: false diff --git a/roles/deploy_container/tasks/action-create.yml b/roles/deploy_container/tasks/action-create.yml index cc06163d..5dadd039 100644 --- a/roles/deploy_container/tasks/action-create.yml +++ b/roles/deploy_container/tasks/action-create.yml @@ -70,7 +70,7 @@ - name: Fetch the aws_acm files. ansible.builtin.get_url: - url: "https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/{{ item }}/main.yml" + url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/aws/aws_acm/{{ item }}/main.yml" dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/{{ item }}/main.yml" delegate_to: localhost with_items: @@ -79,7 +79,7 @@ - name: Fetch the aws_acm tasks. ansible.builtin.get_url: - url: https://raw.githubusercontent.com/codeenigma/ce-provision/1.x/roles/aws/aws_acm/tasks/main.yml + url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/aws/aws_acm/tasks/main.yml dest: "{{ _ce_deploy_base_dir }}/roles/aws_acm/tasks/main.yml" delegate_to: localhost @@ -161,6 +161,9 @@ vpc_id: "{{ _aws_ecs_cluster_vpc_id }}" health_check_protocol: "{{ deploy_container.aws_ecs.health_check.protocol }}" health_check_path: "{{ deploy_container.aws_ecs.health_check.path }}" + health_check_interval: "{{ deploy_container.aws_ecs.health_check.interval | default(omit) }}" + healthy_threshold_count: "{{ deploy_container.aws_ecs.health_check.healthy_threshold_count | default(omit) }}" + unhealthy_threshold_count: "{{ deploy_container.aws_ecs.health_check.unhealthy_threshold_count | default(omit) }}" successful_response_codes: "{{ deploy_container.aws_ecs.health_check.response_codes }}" target_type: ip targets: "{{ deploy_container.aws_ecs.targets }}" From 527f9a057dcb24bcf47bf2a4f550e5447ffb04cf Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 26 Apr 2024 16:30:27 +0200 Subject: [PATCH 127/190] Suspend asg on build pr devel (#464) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- docs/_Sidebar.md | 1 + docs/roles/_exit.md | 2 + docs/roles/_init.md | 41 ++++++++++++------- docs/roles/sync/database_sync.md | 1 + .../sync/database_sync/database_sync-mysql.md | 7 ++++ roles/_exit/README.md | 2 + roles/_exit/tasks/main.yml | 10 ++++- roles/_init/README.md | 41 ++++++++++++------- roles/_init/defaults/main.yml | 29 +++++++------ roles/_init/tasks/main.yml | 10 +++++ roles/sync/database_sync/README.md | 1 + .../database_sync-mysql/README.md | 7 ++++ .../database_sync-mysql/tasks/sync.yml | 1 + 13 files changed, 110 insertions(+), 43 deletions(-) create mode 100644 docs/roles/_exit.md create mode 100644 roles/_exit/README.md diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index a58673b3..baa010a8 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -32,6 +32,7 @@ - [MySQL backups](/roles/database_backup/database_backup-mysql) - [Deploy](/roles/deploy_code) - [Deploy container](/roles/deploy_container) + - [Exit](/roles/_exit) - [Init](/roles/_init) - [LHCI run](/roles/lhci_run) - [Maintenance Mode](/roles/maintenance_mode) diff --git a/docs/roles/_exit.md b/docs/roles/_exit.md new file mode 100644 index 00000000..6aa676a6 --- /dev/null +++ b/docs/roles/_exit.md @@ -0,0 +1,2 @@ +# Exit +Mandatory role that must run after any other `ce-deploy` roles when executing a playbook. diff --git a/docs/roles/_init.md b/docs/roles/_init.md index 98179138..f31e8064 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -1,5 +1,15 @@ # Init -These variables **must** be set in the `deploy/common.yml` file, at least. +Mandatory role that must run before any other `ce-edploy` roles when executing a playbook. + +These variables **must** be set in a common variables file if you do not wish to use defaults. + +In order to manipulate an AWS Autoscaling Group (ASG) your `deploy` user must have an AWS CLI profile for a user with the following IAM permissions: +* `autoscaling:ResumeProcesses` +* `autoscaling:SuspendProcesses` +* `autoscaling:DescribeScalingProcessTypes` +* `autoscaling:DescribeAutoScalingGroups` + +Set the `aws_asg.name` to the machine name of your ASG in order to automatically suspend and resume autoscaling on build. @@ -8,16 +18,24 @@ These variables **must** be set in the `deploy/common.yml` file, at least. ## Default variables ```yaml --- -# Common defaults. Given the "_init" role is mandatory, -# this will ensure defaults to other roles too. -# If you are using ce-provision to deploy infrastructure this must match the `user_deploy.username` variable -deploy_user: "deploy" -# for MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here +# Common defaults - the "_init" role is mandatory so this will ensure defaults to other roles too. +deploy_user: deploy # if you are using ce-provision to deploy infrastructure this must match the `user_deploy.username` variable +# For MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here: _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" +# @TODO only used by Drupal 7, can be removed with Drupal 7 deployments +bin_directory: "/home/{{ deploy_user }}/.bin" +# Number of dumps/db to look up for cleanup. +cleanup_history_depth: 50 +install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app +# AWS ASG variables to allow for the suspension of autoscaling during a code deployment. +aws_asg: + name: "" # if the deploy is on an ASG put the name here + region: "eu-west-1" + suspend_processes: "Launch Terminate" # space separated string, see https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html +# Application specific variables. drupal: drush_verbose_output: false - # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error. - truncate_cache_table: false + truncate_cache_table: false # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error sites: - folder: "default" public_files: "sites/default/files" @@ -37,7 +55,7 @@ drupal: - minute: "*/{{ 10 | random(start=1) }}" job: cron feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment - # For syncing database and files on a feature branch initial build - include all variables if used + # For syncing database and files on a feature branch initial build - include all variables if used: mysql_sync: {} # see sync/database_sync for docs # mysqldump_params: "{{ _mysqldump_params }}" # cleanup: true @@ -50,11 +68,6 @@ drupal: mautic: image_path: "media/images" force_install: false -# Used for custom build time tools like cachetool -bin_directory: "/home/{{ deploy_user }}/.bin" -# Number of dumps/db to look up for cleanup. -cleanup_history_depth: 50 -install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app ``` diff --git a/docs/roles/sync/database_sync.md b/docs/roles/sync/database_sync.md index 850b88ba..d4e8899a 100644 --- a/docs/roles/sync/database_sync.md +++ b/docs/roles/sync/database_sync.md @@ -1,5 +1,6 @@ # Database sync Roles that sync databases between environments. + ## Default variables ```yaml diff --git a/docs/roles/sync/database_sync/database_sync-mysql.md b/docs/roles/sync/database_sync/database_sync-mysql.md index c0c2954b..ab07346d 100644 --- a/docs/roles/sync/database_sync/database_sync-mysql.md +++ b/docs/roles/sync/database_sync/database_sync-mysql.md @@ -1,5 +1,12 @@ # Database sync - MySQL Sync MySQL databases between environments. + +In order to manipulate an AWS Autoscaling Group (ASG) your `deploy` user must have an AWS CLI profile for a user with the following IAM permissions: +* `autoscaling:ResumeProcesses` +* `autoscaling:SuspendProcesses` +* `autoscaling:DescribeScalingProcessTypes` +* `autoscaling:DescribeAutoScalingGroups` + ## Default variables ```yaml diff --git a/roles/_exit/README.md b/roles/_exit/README.md new file mode 100644 index 00000000..6aa676a6 --- /dev/null +++ b/roles/_exit/README.md @@ -0,0 +1,2 @@ +# Exit +Mandatory role that must run after any other `ce-deploy` roles when executing a playbook. diff --git a/roles/_exit/tasks/main.yml b/roles/_exit/tasks/main.yml index 1a62b9af..0ba2da27 100644 --- a/roles/_exit/tasks/main.yml +++ b/roles/_exit/tasks/main.yml @@ -1,3 +1,9 @@ --- - -# We do nothing there, just a placeholder for now. \ No newline at end of file +# If we are operating on an AWS ASG then resume autoscaling. +- name: Enable all autoscale processes on ASG. + ansible.builtin.command: > + aws autoscaling resume-processes --auto-scaling-group-name {{ aws_asg.name }} --region {{ aws_asg.region }} + delegate_to: localhost + when: + - aws_asg.name is defined + - aws_asg.name | length > 0 diff --git a/roles/_init/README.md b/roles/_init/README.md index 98179138..f31e8064 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -1,5 +1,15 @@ # Init -These variables **must** be set in the `deploy/common.yml` file, at least. +Mandatory role that must run before any other `ce-edploy` roles when executing a playbook. + +These variables **must** be set in a common variables file if you do not wish to use defaults. + +In order to manipulate an AWS Autoscaling Group (ASG) your `deploy` user must have an AWS CLI profile for a user with the following IAM permissions: +* `autoscaling:ResumeProcesses` +* `autoscaling:SuspendProcesses` +* `autoscaling:DescribeScalingProcessTypes` +* `autoscaling:DescribeAutoScalingGroups` + +Set the `aws_asg.name` to the machine name of your ASG in order to automatically suspend and resume autoscaling on build. @@ -8,16 +18,24 @@ These variables **must** be set in the `deploy/common.yml` file, at least. ## Default variables ```yaml --- -# Common defaults. Given the "_init" role is mandatory, -# this will ensure defaults to other roles too. -# If you are using ce-provision to deploy infrastructure this must match the `user_deploy.username` variable -deploy_user: "deploy" -# for MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here +# Common defaults - the "_init" role is mandatory so this will ensure defaults to other roles too. +deploy_user: deploy # if you are using ce-provision to deploy infrastructure this must match the `user_deploy.username` variable +# For MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here: _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" +# @TODO only used by Drupal 7, can be removed with Drupal 7 deployments +bin_directory: "/home/{{ deploy_user }}/.bin" +# Number of dumps/db to look up for cleanup. +cleanup_history_depth: 50 +install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app +# AWS ASG variables to allow for the suspension of autoscaling during a code deployment. +aws_asg: + name: "" # if the deploy is on an ASG put the name here + region: "eu-west-1" + suspend_processes: "Launch Terminate" # space separated string, see https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html +# Application specific variables. drupal: drush_verbose_output: false - # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error. - truncate_cache_table: false + truncate_cache_table: false # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error sites: - folder: "default" public_files: "sites/default/files" @@ -37,7 +55,7 @@ drupal: - minute: "*/{{ 10 | random(start=1) }}" job: cron feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment - # For syncing database and files on a feature branch initial build - include all variables if used + # For syncing database and files on a feature branch initial build - include all variables if used: mysql_sync: {} # see sync/database_sync for docs # mysqldump_params: "{{ _mysqldump_params }}" # cleanup: true @@ -50,11 +68,6 @@ drupal: mautic: image_path: "media/images" force_install: false -# Used for custom build time tools like cachetool -bin_directory: "/home/{{ deploy_user }}/.bin" -# Number of dumps/db to look up for cleanup. -cleanup_history_depth: 50 -install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app ``` diff --git a/roles/_init/defaults/main.yml b/roles/_init/defaults/main.yml index 2e4eb0a8..7f06f695 100644 --- a/roles/_init/defaults/main.yml +++ b/roles/_init/defaults/main.yml @@ -1,14 +1,22 @@ --- -# Common defaults. Given the "_init" role is mandatory, -# this will ensure defaults to other roles too. -# If you are using ce-provision to deploy infrastructure this must match the `user_deploy.username` variable -deploy_user: "deploy" -# for MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here +# Common defaults - the "_init" role is mandatory so this will ensure defaults to other roles too. +deploy_user: deploy # if you are using ce-provision to deploy infrastructure this must match the `user_deploy.username` variable +# For MySQL CE you might want to add '--set-gtid-purged=OFF --skip-definer' here: _mysqldump_params: "--max-allowed-packet=128M --single-transaction --skip-opt -e --quick --skip-disable-keys --skip-add-locks -C -a --add-drop-table" +# @TODO only used by Drupal 7, can be removed with Drupal 7 deployments +bin_directory: "/home/{{ deploy_user }}/.bin" +# Number of dumps/db to look up for cleanup. +cleanup_history_depth: 50 +install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app +# AWS ASG variables to allow for the suspension of autoscaling during a code deployment. +aws_asg: + name: "" # if the deploy is on an ASG put the name here + region: "eu-west-1" + suspend_processes: "Launch Terminate" # space separated string, see https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html +# Application specific variables. drupal: drush_verbose_output: false - # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error. - truncate_cache_table: false + truncate_cache_table: false # when set to true - truncate database table cache_container, a workaround to resolve the 'Cannot redeclare ...' error sites: - folder: "default" public_files: "sites/default/files" @@ -28,7 +36,7 @@ drupal: - minute: "*/{{ 10 | random(start=1) }}" job: cron feature_branch: false # whether or not this build is a feature branch that should sync assets from another environment - # For syncing database and files on a feature branch initial build - include all variables if used + # For syncing database and files on a feature branch initial build - include all variables if used: mysql_sync: {} # see sync/database_sync for docs # mysqldump_params: "{{ _mysqldump_params }}" # cleanup: true @@ -41,8 +49,3 @@ drupal: mautic: image_path: "media/images" force_install: false -# Used for custom build time tools like cachetool -bin_directory: "/home/{{ deploy_user }}/.bin" -# Number of dumps/db to look up for cleanup. -cleanup_history_depth: 50 -install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 75523e18..680ba19f 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -122,3 +122,13 @@ ansible.builtin.include_tasks: "{{ project_type }}.yml" when: - _project_type_task_result.stat.exists + +# If we are operating on an AWS ASG then pause autoscaling. +# @TODO - the autoscaling_group module can do this - https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_module.html +- name: Disable autoscale processes on ASG. + ansible.builtin.command: > + aws autoscaling suspend-processes --auto-scaling-group-name {{ aws_asg.name }} --scaling-processes {{ aws_asg.suspend_processes }} --region {{ aws_asg.region }} + delegate_to: localhost + when: + - aws_asg.name is defined + - aws_asg.name | length > 0 diff --git a/roles/sync/database_sync/README.md b/roles/sync/database_sync/README.md index 850b88ba..d4e8899a 100644 --- a/roles/sync/database_sync/README.md +++ b/roles/sync/database_sync/README.md @@ -1,5 +1,6 @@ # Database sync Roles that sync databases between environments. + ## Default variables ```yaml diff --git a/roles/sync/database_sync/database_sync-mysql/README.md b/roles/sync/database_sync/database_sync-mysql/README.md index c0c2954b..ab07346d 100644 --- a/roles/sync/database_sync/database_sync-mysql/README.md +++ b/roles/sync/database_sync/database_sync-mysql/README.md @@ -1,5 +1,12 @@ # Database sync - MySQL Sync MySQL databases between environments. + +In order to manipulate an AWS Autoscaling Group (ASG) your `deploy` user must have an AWS CLI profile for a user with the following IAM permissions: +* `autoscaling:ResumeProcesses` +* `autoscaling:SuspendProcesses` +* `autoscaling:DescribeScalingProcessTypes` +* `autoscaling:DescribeAutoScalingGroups` + ## Default variables ```yaml diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 3f94e47d..41266982 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -17,6 +17,7 @@ - database.target.asg is defined - database.target.asg | length > 0 +# @TODO - the autoscaling_group module can do this - https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_module.html - name: Disable ReplaceUnhealthy autoscale process on source ASG. ansible.builtin.command: > aws autoscaling suspend-processes --auto-scaling-group-name {{ database.source.asg }} --scaling-processes ReplaceUnhealthy --region {{ mysql_sync_source_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} From 17db1db7a7fb830d502c411de346b6f38f896ade Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 26 Apr 2024 16:33:40 +0200 Subject: [PATCH 128/190] Ecs healthcheck vars pr devel (#466) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- docs/roles/deploy_container.md | 4 ++-- roles/_init/README.md | 2 +- roles/deploy_container/README.md | 4 ++-- roles/deploy_container/defaults/main.yml | 4 ++-- roles/deploy_container/tasks/action-create.yml | 4 ++-- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/roles/deploy_container.md b/docs/roles/deploy_container.md index ed42996d..15955a56 100644 --- a/docs/roles/deploy_container.md +++ b/docs/roles/deploy_container.md @@ -88,8 +88,8 @@ deploy_container: - example-cluster-dev-b # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html service_autoscale_metric_type: ECSServiceAverageCPUUtilization - service_autoscale_up_cooldown: 120 - service_autoscale_down_cooldown: 120 + service_autoscale_in_cooldown: 120 # scale down + service_autoscale_out_cooldown: 120 # scale up service_autoscale_target_value: 70 # the value to trigger a scaling event at service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API diff --git a/roles/_init/README.md b/roles/_init/README.md index f31e8064..8d4fa92e 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -1,5 +1,5 @@ # Init -Mandatory role that must run before any other `ce-edploy` roles when executing a playbook. +Mandatory role that must run before any other `ce-deploy` roles when executing a playbook. These variables **must** be set in a common variables file if you do not wish to use defaults. diff --git a/roles/deploy_container/README.md b/roles/deploy_container/README.md index ed42996d..15955a56 100644 --- a/roles/deploy_container/README.md +++ b/roles/deploy_container/README.md @@ -88,8 +88,8 @@ deploy_container: - example-cluster-dev-b # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html service_autoscale_metric_type: ECSServiceAverageCPUUtilization - service_autoscale_up_cooldown: 120 - service_autoscale_down_cooldown: 120 + service_autoscale_in_cooldown: 120 # scale down + service_autoscale_out_cooldown: 120 # scale up service_autoscale_target_value: 70 # the value to trigger a scaling event at service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API diff --git a/roles/deploy_container/defaults/main.yml b/roles/deploy_container/defaults/main.yml index 98b23723..169fb359 100644 --- a/roles/deploy_container/defaults/main.yml +++ b/roles/deploy_container/defaults/main.yml @@ -45,8 +45,8 @@ deploy_container: - example-cluster-dev-b # See docs for values: https://docs.aws.amazon.com/autoscaling/application/APIReference/API_TargetTrackingScalingPolicyConfiguration.html service_autoscale_metric_type: ECSServiceAverageCPUUtilization - service_autoscale_up_cooldown: 120 - service_autoscale_down_cooldown: 120 + service_autoscale_in_cooldown: 120 # scale down + service_autoscale_out_cooldown: 120 # scale up service_autoscale_target_value: 70 # the value to trigger a scaling event at service_public_container_ip: false # set to true to make containers appear on an EIP - more details: https://stackoverflow.com/a/66802973 service_enable_ssm: false # set to true to allow arbitrary command execution on containers via the AWS API diff --git a/roles/deploy_container/tasks/action-create.yml b/roles/deploy_container/tasks/action-create.yml index 5dadd039..58ebf27a 100644 --- a/roles/deploy_container/tasks/action-create.yml +++ b/roles/deploy_container/tasks/action-create.yml @@ -328,8 +328,8 @@ target_tracking_scaling_policy_configuration: PredefinedMetricSpecification: PredefinedMetricType: "{{ deploy_container.aws_ecs.service_autoscale_metric_type }}" - ScaleInCooldown: "{{ deploy_container.aws_ecs.service_autoscale_up_cooldown }}" - ScaleOutCooldown: "{{ deploy_container.aws_ecs.service_autoscale_down_cooldown }}" + ScaleInCooldown: "{{ deploy_container.aws_ecs.service_autoscale_in_cooldown }}" + ScaleOutCooldown: "{{ deploy_container.aws_ecs.service_autoscale_out_cooldown }}" DisableScaleIn: false TargetValue: "{{ deploy_container.aws_ecs.service_autoscale_target_value }}" delegate_to: localhost From 068b0197e7e35b7911377f9ed15f174b54bb9d51 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Thu, 23 May 2024 16:02:43 +0200 Subject: [PATCH 129/190] Bug fixes pr devel (#468) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Moving installation of cachetool to application includes, where they are PHP apps. * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Fixing minor typo. --------- Co-authored-by: EmlynK Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- docs/roles/_init.md | 2 +- roles/database_backup/database_backup-mysql/defaults/main.yml | 2 ++ .../database_backup-mysql/tasks/cleanup-rolling.yml | 1 + .../database_backup/database_backup-mysql/tasks/deploy-dump.yml | 1 + .../database_backup/database_backup-mysql/tasks/deploy-none.yml | 1 + .../database_backup-mysql/tasks/deploy-rolling.yml | 1 + 6 files changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/roles/_init.md b/docs/roles/_init.md index f31e8064..8d4fa92e 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -1,5 +1,5 @@ # Init -Mandatory role that must run before any other `ce-edploy` roles when executing a playbook. +Mandatory role that must run before any other `ce-deploy` roles when executing a playbook. These variables **must** be set in a common variables file if you do not wish to use defaults. diff --git a/roles/database_backup/database_backup-mysql/defaults/main.yml b/roles/database_backup/database_backup-mysql/defaults/main.yml index e7351d89..1401d1ca 100644 --- a/roles/database_backup/database_backup-mysql/defaults/main.yml +++ b/roles/database_backup/database_backup-mysql/defaults/main.yml @@ -5,6 +5,8 @@ mysql_backup: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here # Location on deploy server where the generated MySQL password will be stashed - should be temporary storage mysql_password_path: "/tmp/.ce-deploy/{{ project_name }}_{{ build_type }}_{{ build_number }}" + # Uncomment to login with MySQL socket instead of TCP/IP (e.g. for MariaDB after secure set-up) + #mysql_unix_socket: /run/mysqld/mysqld.sock # Number of dumps/db to keep. Note this is independant from the build codebases. keep: 10 # This can be one of the following: diff --git a/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml index 374e65ca..a94ccc2a 100644 --- a/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/cleanup-rolling.yml @@ -6,5 +6,6 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" + login_unix_socket: "{{ mysql_backup.mysql_unix_socket | default(omit) }}" with_sequence: start={{ [previous_build_number | int - cleanup_history_depth, 0] | max }} end={{ [previous_build_number | int - mysql_backup.keep, 0] | max }} run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml index 49bda27e..28422dd5 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-dump.yml @@ -20,6 +20,7 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" + login_unix_socket: "{{ mysql_backup.mysql_unix_socket | default(omit) }}" when: previous_build_number == 0 or _build_db_status.rc == 1 run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml index 4c4b1624..2a8ef973 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-none.yml @@ -20,5 +20,6 @@ login_host: "{{ _mysql_host }}" login_user: "{{ _mysql_user }}" login_password: "{{ _mysql_password }}" + login_unix_socket: "{{ mysql_backup.mysql_unix_socket | default(omit) }}" when: previous_build_number == 0 or _build_db_status.rc == 1 run_once: true diff --git a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml index 30fbaa62..458e0978 100644 --- a/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml +++ b/roles/database_backup/database_backup-mysql/tasks/deploy-rolling.yml @@ -19,6 +19,7 @@ state: present config_file: "{{ database.credentials_file }}" config_overrides_defaults: true + login_unix_socket: "{{ mysql_backup.mysql_unix_socket | default(omit) }}" run_once: true - name: Check if the previous database exists. From e10bd4b78872379eb07abf8c96ef92a9a8870e23 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 24 May 2024 12:37:04 +0200 Subject: [PATCH 130/190] Lock file pr devel (#470) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Adding pre-flight checks and lock file behaviour to ce-deploy. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Moving lock file paths to variables. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- docs/roles/_init.md | 3 ++ .../database_backup/database_backup-mysql.md | 2 ++ roles/_exit/tasks/main.yml | 5 +++ roles/_init/README.md | 3 ++ roles/_init/defaults/main.yml | 3 ++ roles/_init/tasks/main.yml | 31 +++++++++++++++++++ .../database_backup-mysql/README.md | 2 ++ 7 files changed, 49 insertions(+) diff --git a/docs/roles/_init.md b/docs/roles/_init.md index 8d4fa92e..a1e37f95 100644 --- a/docs/roles/_init.md +++ b/docs/roles/_init.md @@ -27,6 +27,9 @@ bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app +ce_deploy_version: 1.x +lock_file: /tmp/ce-deploy-lock +provision_lock_file: /tmp/ce-provision-lock # must match _init.lock_file in ce-provision # AWS ASG variables to allow for the suspension of autoscaling during a code deployment. aws_asg: name: "" # if the deploy is on an ASG put the name here diff --git a/docs/roles/database_backup/database_backup-mysql.md b/docs/roles/database_backup/database_backup-mysql.md index 2766a4f3..f23c9c34 100644 --- a/docs/roles/database_backup/database_backup-mysql.md +++ b/docs/roles/database_backup/database_backup-mysql.md @@ -24,6 +24,8 @@ mysql_backup: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here # Location on deploy server where the generated MySQL password will be stashed - should be temporary storage mysql_password_path: "/tmp/.ce-deploy/{{ project_name }}_{{ build_type }}_{{ build_number }}" + # Uncomment to login with MySQL socket instead of TCP/IP (e.g. for MariaDB after secure set-up) + #mysql_unix_socket: /run/mysqld/mysqld.sock # Number of dumps/db to keep. Note this is independant from the build codebases. keep: 10 # This can be one of the following: diff --git a/roles/_exit/tasks/main.yml b/roles/_exit/tasks/main.yml index 0ba2da27..640f3a18 100644 --- a/roles/_exit/tasks/main.yml +++ b/roles/_exit/tasks/main.yml @@ -7,3 +7,8 @@ when: - aws_asg.name is defined - aws_asg.name | length > 0 + +- name: Delete the lock file. + ansible.builtin.file: + path: "{{ lock_file }}" + state: absent diff --git a/roles/_init/README.md b/roles/_init/README.md index 8d4fa92e..a1e37f95 100644 --- a/roles/_init/README.md +++ b/roles/_init/README.md @@ -27,6 +27,9 @@ bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app +ce_deploy_version: 1.x +lock_file: /tmp/ce-deploy-lock +provision_lock_file: /tmp/ce-provision-lock # must match _init.lock_file in ce-provision # AWS ASG variables to allow for the suspension of autoscaling during a code deployment. aws_asg: name: "" # if the deploy is on an ASG put the name here diff --git a/roles/_init/defaults/main.yml b/roles/_init/defaults/main.yml index 7f06f695..00c14563 100644 --- a/roles/_init/defaults/main.yml +++ b/roles/_init/defaults/main.yml @@ -8,6 +8,9 @@ bin_directory: "/home/{{ deploy_user }}/.bin" # Number of dumps/db to look up for cleanup. cleanup_history_depth: 50 install_php_cachetool: true # set to false if you don't need cachetool, e.g. for a nodejs app +ce_deploy_version: 1.x +lock_file: /tmp/ce-deploy-lock +provision_lock_file: /tmp/ce-provision-lock # must match _init.lock_file in ce-provision # AWS ASG variables to allow for the suspension of autoscaling during a code deployment. aws_asg: name: "" # if the deploy is on an ASG put the name here diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 680ba19f..1b53f4b2 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -1,4 +1,35 @@ --- +- name: Version check. + ansible.builtin.debug: + msg: "Using ce-deploy {{ ce_deploy_version }}" + +- name: Check for a ce-provision lock file. + ansible.builtin.stat: + path: "{{ provision_lock_file }}" + register: _ce_provision_lock + +- name: Abort if ce-provision lock file exists. + when: _ce_provision_lock.stat.exists is defined and _ce_provision_lock.stat.exists + block: + - name: Abort if ce-provision lock file is found. + ansible.builtin.debug: + msg: "ce-provision lock file discovered, an infrastructure build is in progress! If this is not the case, login to the affected server and delete the file at {{ provision_lock_file }}." + - ansible.builtin.meta: end_play + +- name: Check OS family. + when: ansible_os_family == "Windows" + block: + - name: Abort if target is a Windows server. + ansible.builtin.debug: + msg: "ce-deploy currently only supports Linux like operating systems, and works best with Debian Linux." + - ansible.builtin.meta: end_play + +- name: Set a lock file. + ansible.builtin.file: + path: "{{ lock_file }}" + state: touch + mode: 0644 + # Ensure default values for common variables. - name: Define deploy user. ansible.builtin.set_fact: diff --git a/roles/database_backup/database_backup-mysql/README.md b/roles/database_backup/database_backup-mysql/README.md index 2766a4f3..f23c9c34 100644 --- a/roles/database_backup/database_backup-mysql/README.md +++ b/roles/database_backup/database_backup-mysql/README.md @@ -24,6 +24,8 @@ mysql_backup: mysqldump_params: "{{ _mysqldump_params }}" # set in _init but you can override here # Location on deploy server where the generated MySQL password will be stashed - should be temporary storage mysql_password_path: "/tmp/.ce-deploy/{{ project_name }}_{{ build_type }}_{{ build_number }}" + # Uncomment to login with MySQL socket instead of TCP/IP (e.g. for MariaDB after secure set-up) + #mysql_unix_socket: /run/mysqld/mysqld.sock # Number of dumps/db to keep. Note this is independant from the build codebases. keep: 10 # This can be one of the following: From 603ca5894d6a692334d8b743071d41e7105f9775 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 27 May 2024 18:00:19 +0200 Subject: [PATCH 131/190] Bug fixes pr devel (#475) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Moving installation of cachetool to application includes, where they are PHP apps. * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Fixing minor typo. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. --------- Co-authored-by: EmlynK Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- roles/_meta/deploy-drupal8/tasks/main.yml | 32 +++++++++---------- .../cache_clear-drupal8/tasks/main.yml | 4 +-- roles/config_generate/tasks/main.yml | 6 ++-- .../cron/cron_database_backup/tasks/main.yml | 2 +- roles/cron/cron_drupal8/tasks/cron.yml | 2 +- roles/cron/cron_drupal8/tasks/job.yml | 6 ++++ roles/database_apply/tasks/main.yml | 2 +- .../tasks/offline.yml | 2 ++ .../tasks/online.yml | 2 ++ roles/sanitize/admin_creds/tasks/main.yml | 2 +- 10 files changed, 35 insertions(+), 25 deletions(-) diff --git a/roles/_meta/deploy-drupal8/tasks/main.yml b/roles/_meta/deploy-drupal8/tasks/main.yml index 3e66875b..c8a9223b 100644 --- a/roles/_meta/deploy-drupal8/tasks/main.yml +++ b/roles/_meta/deploy-drupal8/tasks/main.yml @@ -3,41 +3,41 @@ # - single site # - multisites with a single database -- import_role: +- ansible.builtin.import_role: name: _init -- import_role: +- ansible.builtin.import_role: name: deploy_code -- import_role: +- ansible.builtin.import_role: name: composer -- import_role: +- ansible.builtin.import_role: name: npm -- import_role: +- ansible.builtin.import_role: name: maintenance_mode vars: maintenance_mode: operation: offline -- import_role: +- ansible.builtin.import_role: name: database_backup -- import_role: +- ansible.builtin.import_role: name: config_generate -- import_role: +- ansible.builtin.import_role: name: cache_clear/cache_clear-opcache -- import_role: +- ansible.builtin.import_role: name: database_apply -- import_role: +- ansible.builtin.import_role: name: sanitize/admin_creds -- import_role: +- ansible.builtin.import_role: name: live_symlink -- import_role: +- ansible.builtin.import_role: name: cache_clear/cache_clear-drupal8 -- import_role: +- ansible.builtin.import_role: name: maintenance_mode vars: maintenance_mode: operation: online -- import_role: +- ansible.builtin.import_role: name: cron/cron_database_backup -- import_role: +- ansible.builtin.import_role: name: cron/cron_drupal8 -- import_role: +- ansible.builtin.import_role: name: _exit diff --git a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml index a0f960da..063bfe8d 100644 --- a/roles/cache_clear/cache_clear-drupal8/tasks/main.yml +++ b/roles/cache_clear/cache_clear-drupal8/tasks/main.yml @@ -11,7 +11,7 @@ when: - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" - - deploy_operation == 'cleanup' + - deploy_operation == "cleanup" - name: Clear Drupal cache. ansible.builtin.command: @@ -53,4 +53,4 @@ - drupal.truncate_cache_table - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" - - deploy_operation == 'cleanup' + - deploy_operation == "cleanup" diff --git a/roles/config_generate/tasks/main.yml b/roles/config_generate/tasks/main.yml index 2af08e87..adc35b06 100644 --- a/roles/config_generate/tasks/main.yml +++ b/roles/config_generate/tasks/main.yml @@ -2,7 +2,7 @@ - name: Generate project specific configuration. ansible.builtin.include_role: name: "config_generate/config_generate-{{ project_type }}" - when: deploy_operation == 'deploy' + when: deploy_operation == "deploy" - name: Generate additional templates. ansible.builtin.template: @@ -13,7 +13,7 @@ loop_var: template when: - config_generate.templates | length - - deploy_operation == 'deploy' + - deploy_operation == "deploy" - name: Create additional symlinks. ansible.builtin.file: @@ -25,4 +25,4 @@ loop_var: link when: - config_generate.symlinks | length - - deploy_operation == 'deploy' \ No newline at end of file + - deploy_operation == "deploy" \ No newline at end of file diff --git a/roles/cron/cron_database_backup/tasks/main.yml b/roles/cron/cron_database_backup/tasks/main.yml index ab8bc0a1..4cd4676c 100644 --- a/roles/cron/cron_database_backup/tasks/main.yml +++ b/roles/cron/cron_database_backup/tasks/main.yml @@ -6,4 +6,4 @@ with_items: "{{ database_backup.engines }}" loop_control: loop_var: engine - when: deploy_operation == 'deploy' \ No newline at end of file + when: deploy_operation == "deploy" \ No newline at end of file diff --git a/roles/cron/cron_drupal8/tasks/cron.yml b/roles/cron/cron_drupal8/tasks/cron.yml index 1ed40cc3..de153574 100644 --- a/roles/cron/cron_drupal8/tasks/cron.yml +++ b/roles/cron/cron_drupal8/tasks/cron.yml @@ -6,4 +6,4 @@ loop_control: loop_var: entry when: - - deploy_operation == 'deploy' + - (deploy_operation == "deploy") or (deploy_operation == "revert") diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index b89a0b05..7fa23c3e 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -2,6 +2,12 @@ - name: Define cron job command. ansible.builtin.set_fact: _cron_job_command: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }}" + when: deploy_operation == "deploy" + +- name: Revert cron job command. + ansible.builtin.set_fact: + _cron_job_command: "cd {{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }}" + when: deploy_operation == "revert" - name: Define cron job command if deferred (ASG). ansible.builtin.set_fact: diff --git a/roles/database_apply/tasks/main.yml b/roles/database_apply/tasks/main.yml index 89138a98..ae1ad8ee 100644 --- a/roles/database_apply/tasks/main.yml +++ b/roles/database_apply/tasks/main.yml @@ -2,5 +2,5 @@ - name: Apply database updates. include_role: name: "database_apply/database_apply-{{ project_type }}" - when: deploy_operation == 'deploy' + when: deploy_operation == "deploy" run_once: true diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml index 78314024..d70f87d0 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/offline.yml @@ -9,6 +9,7 @@ when: - previous_build_number > 0 - project_type == 'drupal8' + - deploy_operation == "deploy" # skip on cleanup and revert, no need # For some reason D7 drush doesn't respect 'chdir' with command, using shell instead. - name: Enable maintenance mode D7. @@ -20,3 +21,4 @@ when: - previous_build_number > 0 - project_type == 'drupal7' + - deploy_operation == "deploy" diff --git a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml index 47a925f6..02646ff6 100644 --- a/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml +++ b/roles/maintenance_mode/maintenance_mode-drupal-core/tasks/online.yml @@ -21,6 +21,7 @@ - project_type == "drupal8" - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" + - deploy_operation == "deploy" # For some reason D7 drush doesn't respect 'chdir' with command, using shell instead. - name: Disable maintenance mode D7. @@ -43,3 +44,4 @@ - project_type == "drupal7" - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" + - deploy_operation == "deploy" diff --git a/roles/sanitize/admin_creds/tasks/main.yml b/roles/sanitize/admin_creds/tasks/main.yml index 880844ba..bc6f511d 100644 --- a/roles/sanitize/admin_creds/tasks/main.yml +++ b/roles/sanitize/admin_creds/tasks/main.yml @@ -2,5 +2,5 @@ - name: Revert admin credentials. include_role: name: "sanitize/admin_creds/admin_creds-{{ project_type }}" - when: deploy_operation == 'deploy' + when: deploy_operation == "deploy" run_once: true From 63301bb13d7cda94134774e5011e39a060a223e2 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 27 May 2024 18:13:59 +0200 Subject: [PATCH 132/190] Drupal 10 meta pr devel (#477) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- docs/_Sidebar.md | 1 + docs/roles/_meta/deploy-drupal10.md | 19 +++++++++++ roles/_meta/deploy-drupal10/README.md | 19 +++++++++++ roles/_meta/deploy-drupal10/tasks/main.yml | 39 ++++++++++++++++++++++ 4 files changed, 78 insertions(+) create mode 100644 docs/roles/_meta/deploy-drupal10.md create mode 100644 roles/_meta/deploy-drupal10/README.md create mode 100644 roles/_meta/deploy-drupal10/tasks/main.yml diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index baa010a8..b8c4d52e 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -37,6 +37,7 @@ - [LHCI run](/roles/lhci_run) - [Maintenance Mode](/roles/maintenance_mode) - ["Meta"](/roles/_meta) + - [Drupal 10](/roles/_meta/deploy-drupal10) - [Drupal 7](/roles/_meta/deploy-drupal7) - [Drupal 8](/roles/_meta/deploy-drupal8) - [Matomo](/roles/_meta/deploy-matomo) diff --git a/docs/roles/_meta/deploy-drupal10.md b/docs/roles/_meta/deploy-drupal10.md new file mode 100644 index 00000000..03cd9e40 --- /dev/null +++ b/docs/roles/_meta/deploy-drupal10.md @@ -0,0 +1,19 @@ +# Drupal 10 +Role for deploying single Drupal instances, or multisites with a single database, of Drupal version 10.0.0 or higher. + +You should always top and tail this role with the `_init` and `_exit` roles and include the files with your Ansible variable overrides in via `vars_files`, here is an example playbook: + +```yaml +--- +- hosts: web1.example.com + vars_files: + - vars/common.yml + - vars/prod.yml + roles: + - _init + - _meta/deploy-drupal10 + - _exit +``` + + + diff --git a/roles/_meta/deploy-drupal10/README.md b/roles/_meta/deploy-drupal10/README.md new file mode 100644 index 00000000..03cd9e40 --- /dev/null +++ b/roles/_meta/deploy-drupal10/README.md @@ -0,0 +1,19 @@ +# Drupal 10 +Role for deploying single Drupal instances, or multisites with a single database, of Drupal version 10.0.0 or higher. + +You should always top and tail this role with the `_init` and `_exit` roles and include the files with your Ansible variable overrides in via `vars_files`, here is an example playbook: + +```yaml +--- +- hosts: web1.example.com + vars_files: + - vars/common.yml + - vars/prod.yml + roles: + - _init + - _meta/deploy-drupal10 + - _exit +``` + + + diff --git a/roles/_meta/deploy-drupal10/tasks/main.yml b/roles/_meta/deploy-drupal10/tasks/main.yml new file mode 100644 index 00000000..6b0d86ab --- /dev/null +++ b/roles/_meta/deploy-drupal10/tasks/main.yml @@ -0,0 +1,39 @@ +--- +# Default role for Drupal 10 and upwards. This is suitable for: +# - single site +# - multisites with a single database + +- ansible.builtin.import_role: + name: deploy_code +- ansible.builtin.import_role: + name: composer +- ansible.builtin.import_role: + name: npm +- ansible.builtin.import_role: + name: maintenance_mode + vars: + maintenance_mode: + operation: offline +- ansible.builtin.import_role: + name: database_backup +- ansible.builtin.import_role: + name: config_generate +- ansible.builtin.import_role: + name: cache_clear/cache_clear-opcache +- ansible.builtin.import_role: + name: database_apply +- ansible.builtin.import_role: + name: sanitize/admin_creds +- ansible.builtin.import_role: + name: live_symlink +- ansible.builtin.import_role: + name: cache_clear/cache_clear-drupal8 +- ansible.builtin.import_role: + name: maintenance_mode + vars: + maintenance_mode: + operation: online +- ansible.builtin.import_role: + name: cron/cron_database_backup +- ansible.builtin.import_role: + name: cron/cron_drupal8 From bc5f5622668bbc407cb6d7457ca2d23f8a92b934 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 31 May 2024 10:12:37 +0300 Subject: [PATCH 133/190] R69347 db import from deploy server pr devel (#479) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server * fix conflicts --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- .../database_sync-mysql/defaults/main.yml | 4 +- .../database_sync-mysql/tasks/sync.yml | 211 ++++++++++-------- 2 files changed, 119 insertions(+), 96 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index 98ac19bf..84f38e42 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -13,9 +13,11 @@ mysql_sync: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" # This can be of types: # - rolling: (database backups). In that case we'll need build parameters.@todo - # - fixed: "fixed" database name + # - fixed: "fixed" database name # currently this var means the same as "dump" # - dump: Use an existing dump. In that case, the "database" variable is the absolute file path. + # This parameter is ignored if var 'path_on_deploy_server' is defined. type: fixed + #path_on_deploy_server: "" # full path to the compressed dump file on the deploy server, overrides "type" variable for source DB when defined. # For "rolling builds", so we can compute the database name. build_id: mybuildprod # Whether or not use to create a fresh database backup or use a nightly one. diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 41266982..fbf82ade 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -1,42 +1,38 @@ --- -- name: Get database source host region. - amazon.aws.ec2_metadata_facts: - register: mysql_sync_source_database_host_info - delegate_to: "{{ database.source.host }}" - when: - - database.source.asg is defined - - database.source.asg | length > 0 - - database.source.fresh_db is defined - - database.source.fresh_db - -# This task does not need a delegate_to because the hosts set in the sync playbook in the repo should be the target host. -- name: Get database target host region. - amazon.aws.ec2_metadata_facts: - register: mysql_sync_target_database_host_info - when: - - database.target.asg is defined - - database.target.asg | length > 0 - -# @TODO - the autoscaling_group module can do this - https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_module.html -- name: Disable ReplaceUnhealthy autoscale process on source ASG. - ansible.builtin.command: > - aws autoscaling suspend-processes --auto-scaling-group-name {{ database.source.asg }} --scaling-processes ReplaceUnhealthy --region {{ mysql_sync_source_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} - delegate_to: localhost +- name: Prepare source ASG. + block: + - name: Get database source host region. + amazon.aws.ec2_metadata_facts: + register: mysql_sync_source_database_host_info + delegate_to: "{{ database.source.host }}" + + # @TODO - the autoscaling_group module can do this - https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_module.html + - name: Disable ReplaceUnhealthy autoscale process on source ASG. + ansible.builtin.command: > + aws autoscaling suspend-processes --auto-scaling-group-name {{ database.source.asg }} --scaling-processes ReplaceUnhealthy --region {{ mysql_sync_source_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} + delegate_to: localhost when: - database.source.asg is defined - database.source.asg | length > 0 - database.source.fresh_db is defined - database.source.fresh_db - -- name: Disable ReplaceUnhealthy autoscale process on target ASG. - ansible.builtin.command: > - aws autoscaling suspend-processes --auto-scaling-group-name {{ database.target.asg }} --scaling-processes ReplaceUnhealthy --region {{ mysql_sync_target_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} - delegate_to: localhost + - database.source.path_on_deploy_server is not defined + +- name: Prepare target ASG. + block: + # This task does not need a delegate_to because the hosts set in the sync playbook in the repo should be the target host. + - name: Get database target host region. + amazon.aws.ec2_metadata_facts: + register: mysql_sync_target_database_host_info + + - name: Disable ReplaceUnhealthy autoscale process on target ASG. + ansible.builtin.command: > + aws autoscaling suspend-processes --auto-scaling-group-name {{ database.target.asg }} --scaling-processes ReplaceUnhealthy --region {{ mysql_sync_target_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} + delegate_to: localhost when: - database.target.asg is defined - database.target.asg | length > 0 - - name: Register bzip2 archive type vars. ansible.builtin.set_fact: archive_file_type: "bz2" @@ -54,6 +50,8 @@ - name: Register remote dump name (from database). ansible.builtin.set_fact: mysql_sync_source_dump_path: "/tmp/{{ database.source.database }}_{{ build_number }}_source.sql.{{ archive_file_type }}" + when: + - database.source.path_on_deploy_server is not defined - name: Get source last known good build number. ansible.builtin.command: @@ -64,46 +62,60 @@ - "{{ database.source.build_id }}" register: mysql_sync_source_build_number delegate_to: localhost - when: database.source.type == 'rolling' + when: + - database.source.type == 'rolling' - name: Register source database name. ansible.builtin.set_fact: mysql_sync_source_database: "{{ database.source.database }}_{{ mysql_sync_source_build_number.stdout }}" - when: database.source.type == 'rolling' + when: + - database.source.type == 'rolling' - name: Register source database name. ansible.builtin.set_fact: mysql_sync_source_database: "{{ database.source.database }}" - when: not database.source.type == 'rolling' - -- name: Take a dump from source database. - ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | {{ archival_command }} > {{ mysql_sync_source_dump_path }}" - args: - executable: /bin/bash - delegate_to: "{{ database.source.host }}" when: - - database.source.fresh_db - -- name: Find source database host. - ansible.builtin.command: - cmd: "grep 'host' {{ database.source.credentials_file }}" - register: mysql_host_info_grep - delegate_to: "{{ database.source.host }}" - when: not database.source.fresh_db - -- name: Register source database host. - set_fact: - mysql_sync_source_database_host: "{{ mysql_host_info_grep.stdout.split('=')[1] }}" - delegate_to: "{{ database.source.host }}" - when: not database.source.fresh_db - -- name: Copy a nightly backup for the source database. - ansible.builtin.copy: - src: "{{ database.source.dumps_directory }}/{{ mysql_sync_source_database_host }}/{{ database.source.database }}" - dest: "{{ mysql_sync_source_dump_path }}" - remote_src: true + - not database.source.type == 'rolling' + - database.source.path_on_deploy_server is not defined + +- name: Fetch the source database. + block: + - name: Take a dump from source database. + ansible.builtin.shell: "set -o pipefail && mysqldump --defaults-extra-file={{ database.source.credentials_file }} {{ mysql_sync.mysqldump_params }} {{ mysql_sync_source_database }} | {{ archival_command }} > {{ mysql_sync_source_dump_path }}" + args: + executable: /bin/bash + when: + - database.source.fresh_db + + - name: Find source database host. + ansible.builtin.command: + cmd: "grep 'host' {{ database.source.credentials_file }}" + register: mysql_host_info_grep + when: + - not database.source.fresh_db + + - name: Register source database host. + set_fact: + mysql_sync_source_database_host: "{{ mysql_host_info_grep.stdout.split('=')[1] }}" + when: + - not database.source.fresh_db + + - name: Copy a nightly backup for the source database. + ansible.builtin.copy: + src: "{{ database.source.dumps_directory }}/{{ mysql_sync_source_database_host }}/{{ database.source.database }}" + dest: "{{ mysql_sync_source_dump_path }}" + remote_src: true + when: + - not database.source.fresh_db + + - name: Fetch dump file. + ansible.builtin.fetch: + src: "{{ mysql_sync_source_dump_path }}" + dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.{{ archive_file_type }}" + flat: true delegate_to: "{{ database.source.host }}" - when: not database.source.fresh_db + when: + - database.source.path_on_deploy_server is not defined - name: Register tmp target dump name. ansible.builtin.set_fact: @@ -122,29 +134,34 @@ - "{{ database.target.build_id }}" register: mysql_sync_target_build_number delegate_to: localhost - when: database.target.type == 'rolling' + when: + - database.target.type == 'rolling' - name: Register target rolling database name. ansible.builtin.set_fact: mysql_sync_target_database: "{{ database.target.database }}_{{ mysql_sync_target_build_number.stdout }}" - when: database.target.type == 'rolling' + when: + - database.target.type == 'rolling' - name: Register target static database name. ansible.builtin.set_fact: mysql_sync_target_database: "{{ database.target.database }}" - when: not database.target.type == 'rolling' - -- name: Fetch dump file. - ansible.builtin.fetch: - src: "{{ mysql_sync_source_dump_path }}" - dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.{{ archive_file_type }}" - flat: true - delegate_to: "{{ database.source.host }}" + when: + - not database.target.type == 'rolling' - name: Copy dump file to destination. ansible.builtin.copy: src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.{{ archive_file_type }}" dest: "{{ mysql_sync_target_dump_path }}" + when: + - database.source.path_on_deploy_server is not defined + +- name: Copy dump file from deploy server to destination. + ansible.builtin.copy: + src: "{{ database.source.path_on_deploy_server }}" + dest: "{{ mysql_sync_target_dump_path }}" + when: + - database.source.path_on_deploy_server is defined - name: Unpack dump file. ansible.builtin.shell: "{{ archival_command }} -d -c {{ mysql_sync_target_dump_path }} > {{ mysql_sync_target_dump_unpacked_path }}" @@ -174,32 +191,36 @@ path: "{{ mysql_sync_target_dump_unpacked_path }}" state: absent -- name: Delete temporary dump file on source. - ansible.builtin.file: - path: "{{ mysql_sync_source_dump_path }}" - state: absent - delegate_to: "{{ database.source.host }}" - -- name: Delete temporary dump file on deploy server. - ansible.builtin.file: - path: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql{{ item }}" - state: absent - delegate_to: localhost - when: - - mysql_sync.cleanup - with_items: - - ".bz2" - - ".gz" - -- name: Enable all autoscale processes on source ASG. - ansible.builtin.command: > - aws autoscaling resume-processes --auto-scaling-group-name {{ database.source.asg }} --region {{ mysql_sync_source_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} - delegate_to: localhost - when: - - database.source.asg is defined - - database.source.asg | length > 0 - - database.source.fresh_db is defined - - database.source.fresh_db +- name: Clean up source. + block: + - name: Delete temporary dump file on source. + ansible.builtin.file: + path: "{{ mysql_sync_source_dump_path }}" + state: absent + delegate_to: "{{ database.source.host }}" + + - name: Delete temporary dump file on deploy server. + ansible.builtin.file: + path: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql{{ item }}" + state: absent + delegate_to: localhost + when: + - mysql_sync.cleanup + with_items: + - ".bz2" + - ".gz" + + - name: Enable all autoscale processes on source ASG. + ansible.builtin.command: > + aws autoscaling resume-processes --auto-scaling-group-name {{ database.source.asg }} --region {{ mysql_sync_source_database_host_info.ansible_facts.ansible_ec2_instance_identity_document_region }} + delegate_to: localhost + when: + - database.source.asg is defined + - database.source.asg | length > 0 + - database.source.fresh_db is defined + - database.source.fresh_db + when: + - database.source.path_on_deploy_server is not defined - name: Enable all autoscale processes on target ASG. ansible.builtin.command: > From a771d55361a59571017e0e5b284fec84e0373e45 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 31 May 2024 13:02:55 +0300 Subject: [PATCH 134/190] R69347 db import from deploy server pr devel (#481) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server * fix conflicts * r69347-db-import-from-deploy-server --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- .../database_sync-mysql/tasks/sync.yml | 29 ++++++++++--------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index fbf82ade..01f2ca52 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -53,23 +53,24 @@ when: - database.source.path_on_deploy_server is not defined -- name: Get source last known good build number. - ansible.builtin.command: - argv: - - "/bin/sh" - - "{{ _ce_deploy_base_dir }}/scripts/track-get.sh" - - "--build-id" - - "{{ database.source.build_id }}" - register: mysql_sync_source_build_number - delegate_to: localhost - when: - - database.source.type == 'rolling' +- name: Detect rolling DB name on source. + block: + - name: Get source last known good build number. + ansible.builtin.command: + argv: + - "/bin/sh" + - "{{ _ce_deploy_base_dir }}/scripts/track-get.sh" + - "--build-id" + - "{{ database.source.build_id }}" + register: mysql_sync_source_build_number + delegate_to: localhost -- name: Register source database name. - ansible.builtin.set_fact: - mysql_sync_source_database: "{{ database.source.database }}_{{ mysql_sync_source_build_number.stdout }}" + - name: Register source database name. + ansible.builtin.set_fact: + mysql_sync_source_database: "{{ database.source.database }}_{{ mysql_sync_source_build_number.stdout }}" when: - database.source.type == 'rolling' + - database.source.path_on_deploy_server is not defined - name: Register source database name. ansible.builtin.set_fact: From 9ef0a10e9930705e34d1ceadc02fec39c3c566ce Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 31 May 2024 14:17:48 +0300 Subject: [PATCH 135/190] R69347 db import from deploy server pr devel (#482) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server * fix conflicts * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- roles/sync/database_sync/database_sync-mysql/tasks/sync.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 01f2ca52..3904650a 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -69,6 +69,7 @@ ansible.builtin.set_fact: mysql_sync_source_database: "{{ database.source.database }}_{{ mysql_sync_source_build_number.stdout }}" when: + - database.source.type is defined - database.source.type == 'rolling' - database.source.path_on_deploy_server is not defined @@ -76,6 +77,7 @@ ansible.builtin.set_fact: mysql_sync_source_database: "{{ database.source.database }}" when: + - database.source.type is defined - not database.source.type == 'rolling' - database.source.path_on_deploy_server is not defined From 12032f764a20d3a1bf473e27eb9724bbf197c230 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Fri, 31 May 2024 17:54:32 +0300 Subject: [PATCH 136/190] R69347 db import from deploy server pr devel (#483) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server * fix conflicts * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- .../database_sync/database_sync-mysql/defaults/main.yml | 4 +++- .../sync/database_sync/database_sync-mysql/tasks/sync.yml | 8 ++++---- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml index 84f38e42..359f9e55 100644 --- a/roles/sync/database_sync/database_sync-mysql/defaults/main.yml +++ b/roles/sync/database_sync/database_sync-mysql/defaults/main.yml @@ -17,7 +17,6 @@ mysql_sync: # - dump: Use an existing dump. In that case, the "database" variable is the absolute file path. # This parameter is ignored if var 'path_on_deploy_server' is defined. type: fixed - #path_on_deploy_server: "" # full path to the compressed dump file on the deploy server, overrides "type" variable for source DB when defined. # For "rolling builds", so we can compute the database name. build_id: mybuildprod # Whether or not use to create a fresh database backup or use a nightly one. @@ -27,6 +26,9 @@ mysql_sync: dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/regular" # If the source is on an ASG, provide the ASG name here. Otherwise, leave empty. asg: "" + # Uncomment and specify full path if you want to use your own dump file on the deploy server (gzip compressed). + # In this case it's the only variable you need in 'database.source'. + #path_on_deploy_server: "/home/{{ deploy_user }}/compressed_database_dump.gz" target: database: "{{ project_name }}_dev" credentials_file: "/home/{{ deploy_user }}/.mysql.creds" diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 3904650a..0ae03be9 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -117,7 +117,7 @@ dest: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.{{ archive_file_type }}" flat: true delegate_to: "{{ database.source.host }}" - when: + when: - database.source.path_on_deploy_server is not defined - name: Register tmp target dump name. @@ -156,14 +156,14 @@ ansible.builtin.copy: src: "{{ _ce_deploy_build_tmp_dir }}/{{ database.target.database }}.sql.{{ archive_file_type }}" dest: "{{ mysql_sync_target_dump_path }}" - when: + when: - database.source.path_on_deploy_server is not defined - name: Copy dump file from deploy server to destination. ansible.builtin.copy: src: "{{ database.source.path_on_deploy_server }}" dest: "{{ mysql_sync_target_dump_path }}" - when: + when: - database.source.path_on_deploy_server is defined - name: Unpack dump file. @@ -222,7 +222,7 @@ - database.source.asg | length > 0 - database.source.fresh_db is defined - database.source.fresh_db - when: + when: - database.source.path_on_deploy_server is not defined - name: Enable all autoscale processes on target ASG. From 4994532a367f308b1251178fc2d13499881142ff Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 5 Jun 2024 10:21:34 +0200 Subject: [PATCH 137/190] Bug fixes pr devel (#485) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Moving installation of cachetool to application includes, where they are PHP apps. * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Fixing minor typo. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. --------- Co-authored-by: Emlyn Kinzett Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton --- roles/_init/tasks/main.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 1b53f4b2..6f0cd830 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -89,10 +89,6 @@ ansible.builtin.set_fact: live_symlink_build_dest: "{{ live_symlink_build_dest | default('{{ build_base_path }}/live.{{ project_name }}_{{ build_type }}') }}" - - name: Set the previous deploy's path for later use where we need to manipulate the live site. - ansible.builtin.set_fact: - previous_deploy_path: "{{ previous_deploy_path | default('{{ deploy_path_prefix }}{{ previous_build_number }}') }}" - - name: Overwrite deploy and live_symlink paths if SquashFS deploy. ansible.builtin.set_fact: deploy_path: "{{ build_path | default('{{ build_path_prefix }}{{ build_number }}') }}" @@ -111,6 +107,10 @@ - deploy_code.mount_type is defined - deploy_code.mount_type == "squashfs" +- name: Set the previous deploy's path for later use where we need to manipulate the live site. + ansible.builtin.set_fact: + previous_deploy_path: "{{ previous_deploy_path | default('{{ deploy_path_prefix }}{{ previous_build_number }}') }}" + # Gather last known good build directly from symlink. # This can happen: # - when the first builds failed, From 4bb291b96af8860d6f9fa6a93d6823cf6b9a4635 Mon Sep 17 00:00:00 2001 From: Chris Maiden Date: Thu, 6 Jun 2024 11:05:46 +0100 Subject: [PATCH 138/190] 472 Cache clearing for Symfony based projects (#484) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable --- roles/cache_clear/cache_clear-mautic/tasks/main.yml | 2 -- roles/cache_clear/cache_clear-symfony/README.md | 7 +++++++ .../cache_clear/cache_clear-symfony/defaults/main.yml | 2 ++ roles/cache_clear/cache_clear-symfony/tasks/main.yml | 10 ++++++++++ 4 files changed, 19 insertions(+), 2 deletions(-) create mode 100644 roles/cache_clear/cache_clear-symfony/README.md create mode 100644 roles/cache_clear/cache_clear-symfony/defaults/main.yml create mode 100644 roles/cache_clear/cache_clear-symfony/tasks/main.yml diff --git a/roles/cache_clear/cache_clear-mautic/tasks/main.yml b/roles/cache_clear/cache_clear-mautic/tasks/main.yml index 9ecec98c..56884d16 100644 --- a/roles/cache_clear/cache_clear-mautic/tasks/main.yml +++ b/roles/cache_clear/cache_clear-mautic/tasks/main.yml @@ -2,11 +2,9 @@ - name: set console path set_fact: mautic_console_bin: "env php {{ deploy_path }}/{{ webroot }}/app/console" - run_once: true - name: Clear cache. command: cmd: "{{ mautic_console_bin }} cache:clear --no-interaction --env=prod --no-warmup" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" - run_once: true diff --git a/roles/cache_clear/cache_clear-symfony/README.md b/roles/cache_clear/cache_clear-symfony/README.md new file mode 100644 index 00000000..8bb3fde2 --- /dev/null +++ b/roles/cache_clear/cache_clear-symfony/README.md @@ -0,0 +1,7 @@ +# Symfony +Clears the Symfony application cache. + + + + + diff --git a/roles/cache_clear/cache_clear-symfony/defaults/main.yml b/roles/cache_clear/cache_clear-symfony/defaults/main.yml new file mode 100644 index 00000000..c90f2d13 --- /dev/null +++ b/roles/cache_clear/cache_clear-symfony/defaults/main.yml @@ -0,0 +1,2 @@ +--- +cache_clear_symfony_env: prod diff --git a/roles/cache_clear/cache_clear-symfony/tasks/main.yml b/roles/cache_clear/cache_clear-symfony/tasks/main.yml new file mode 100644 index 00000000..689af2c2 --- /dev/null +++ b/roles/cache_clear/cache_clear-symfony/tasks/main.yml @@ -0,0 +1,10 @@ +--- +- name: Set console path. + ansible.builtin.set_fact: + symfony_console_bin: "{{ symfony_console_bin | default('env php ' + deploy_path + '/bin/console') }}" + +- name: Clear cache. + ansible.builtin.command: + cmd: "{{ symfony_console_bin }} cache:clear --no-interaction --env={{ cache_clear_symfony_env }} --no-warmup" + become: "{{ 'no' if www_user == deploy_user else 'yes' }}" + become_user: "{{ www_user }}" From 494cf0d8e518babca5fefdf147b6dc7972459ad2 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 19 Jul 2024 13:31:31 +0200 Subject: [PATCH 139/190] Feature branching pr devel (#487) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. * Adding syncing for feature branches. (#346) * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * First pass at NGINX vhost handling in ce-deploy. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/defaults/main.yml | 8 ++++ roles/deploy_code/tasks/deploy.yml | 65 +++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+) diff --git a/roles/deploy_code/defaults/main.yml b/roles/deploy_code/defaults/main.yml index 7b68c3a7..b8220475 100644 --- a/roles/deploy_code/defaults/main.yml +++ b/roles/deploy_code/defaults/main.yml @@ -1,5 +1,13 @@ --- +# Required for NGINX config in feature branching. +_ce_provision_data_dir: /home/deploy/ce-deploy/data + deploy_code: + # Feature branching config. + feature_branch: + enabled: false + # NGINX domains to build, see ce-provision for docs: https://github.com/codeenigma/ce-provision/blob/2.x/roles/debian/nginx/defaults/main.yml#L133 + domains: [] # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 0d15425c..95e3a310 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -38,3 +38,68 @@ loop_var: link when: - deploy_code.symlinks | length + +# Additional vhost handling for feature branch builds. + +# Fetch the NGINX domain handling tasks from ce-provision. +- name: Ensure the nginx directory exists. + ansible.builtin.file: + path: "{{ _ce_deploy_base_dir }}/roles/nginx/{{ item }}" + state: directory + mode: '0755' + delegate_to: localhost + with_items: + - tasks + - defaults + - templates + when: deploy_code.feature_branch.enabled + +- name: Fetch the nginx role files from ce-provision. + when: deploy_code.feature_branch.enabled + delegate_to: localhost + block: + - name: Fetch domain.yml. + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/domain.yml + dest: "{{ _ce_deploy_base_dir }}/roles/nginx/tasks/domain.yml" + + - name: Fetch defaults. + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/defaults/main.yml + dest: "{{ _ce_deploy_base_dir }}/roles/nginx/defaults/main.yml" + + - name: Fetch templates. + ansible.builtin.get_url: + url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/templates/{{ item }}" + dest: "{{ _ce_deploy_base_dir }}/roles/nginx/templates/{{ item }}" + with_items: + - vhosts.j2 + - cloudwatch-vhost.json.j2 + +- name: Create vhost. + when: + - deploy_code.feature_branch.domains is defined + - deploy_code.feature_branch.domains | length > 0 + - deploy_code.feature_branch.enabled + become: true + block: + - name: Generate domain specific configuration. + ansible.builtin.include_tasks: "{{ _ce_deploy_base_dir }}/roles/nginx/tasks/domain.yml" + with_items: "{{ deploy_code.feature_branch.domains }}" + loop_control: + loop_var: domain + + - name: Test NGINX configuration. + ansible.builtin.command: nginx -t + register: _nginx_test_result + failed_when: false + + - name: Display NGINX test result. + ansible.builtin.debug: + msg: "{{ _nginx_test_result.stderr }}" + + - name: Ensure NGINX is reloaded. + ansible.builtin.service: + name: nginx + state: reloaded + when: _nginx_test_result.rc == 0 From 70b13139ad3723f58defb5bb6658ae4ea9ae4524 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 19 Jul 2024 13:49:37 +0200 Subject: [PATCH 140/190] Feature branching pr devel (#489) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. * Adding syncing for feature branches. (#346) * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/tasks/deploy.yml | 66 +++++++++++++++++++++++------- 1 file changed, 51 insertions(+), 15 deletions(-) diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 95e3a310..9e62c41c 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -42,33 +42,43 @@ # Additional vhost handling for feature branch builds. # Fetch the NGINX domain handling tasks from ce-provision. -- name: Ensure the nginx directory exists. - ansible.builtin.file: - path: "{{ _ce_deploy_base_dir }}/roles/nginx/{{ item }}" - state: directory - mode: '0755' - delegate_to: localhost - with_items: - - tasks - - defaults - - templates - when: deploy_code.feature_branch.enabled - - name: Fetch the nginx role files from ce-provision. when: deploy_code.feature_branch.enabled delegate_to: localhost block: - - name: Fetch domain.yml. + - name: Ensure the nginx directory exists. + ansible.builtin.file: + path: "{{ _ce_deploy_base_dir }}/roles/nginx/{{ item }}" + state: directory + mode: '0755' + delegate_to: localhost + with_items: + - tasks + - defaults + - templates + + - name: Ensure the ssl directory exists. + ansible.builtin.file: + path: "{{ _ce_deploy_base_dir }}/roles/ssl/{{ item }}" + state: directory + mode: '0755' + delegate_to: localhost + with_items: + - tasks + - defaults + - templates + + - name: Fetch nginx domain.yml. ansible.builtin.get_url: url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/domain.yml dest: "{{ _ce_deploy_base_dir }}/roles/nginx/tasks/domain.yml" - - name: Fetch defaults. + - name: Fetch nginx defaults. ansible.builtin.get_url: url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/defaults/main.yml dest: "{{ _ce_deploy_base_dir }}/roles/nginx/defaults/main.yml" - - name: Fetch templates. + - name: Fetch nginx templates. ansible.builtin.get_url: url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/templates/{{ item }}" dest: "{{ _ce_deploy_base_dir }}/roles/nginx/templates/{{ item }}" @@ -76,6 +86,32 @@ - vhosts.j2 - cloudwatch-vhost.json.j2 + - name: Fetch ssl tasks. + ansible.builtin.get_url: + url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/ssl/tasks/{{ item }}.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/ssl/tasks/{{ item }}.yml" + with_items: + - copy + - generate + - letsencrypt + - main + - manual + - selfsigned + - unmanaged + + - name: Fetch ssl defaults. + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/ssl/defaults/main.yml + dest: "{{ _ce_deploy_base_dir }}/roles/ssl/defaults/main.yml" + + - name: Fetch ssl templates. + ansible.builtin.get_url: + url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/ssl/templates/{{ item }}" + dest: "{{ _ce_deploy_base_dir }}/roles/nginx/ssl/{{ item }}" + with_items: + - le_cron.sh.j2 + +# Generate the NGINX vhost. - name: Create vhost. when: - deploy_code.feature_branch.domains is defined From a493f6a816d847d0e25639da413ca7b70e1b383e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 19 Jul 2024 13:55:25 +0200 Subject: [PATCH 141/190] Feature branching pr devel (#490) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. * Adding syncing for feature branches. (#346) * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/tasks/deploy.yml | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 9e62c41c..312ae8b3 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -48,7 +48,7 @@ block: - name: Ensure the nginx directory exists. ansible.builtin.file: - path: "{{ _ce_deploy_base_dir }}/roles/nginx/{{ item }}" + path: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/{{ item }}" state: directory mode: '0755' delegate_to: localhost @@ -59,7 +59,7 @@ - name: Ensure the ssl directory exists. ansible.builtin.file: - path: "{{ _ce_deploy_base_dir }}/roles/ssl/{{ item }}" + path: "{{ _ce_deploy_base_dir }}/roles/debian/ssl/{{ item }}" state: directory mode: '0755' delegate_to: localhost @@ -68,20 +68,23 @@ - defaults - templates - - name: Fetch nginx domain.yml. + - name: Fetch nginx tasks. ansible.builtin.get_url: - url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/domain.yml - dest: "{{ _ce_deploy_base_dir }}/roles/nginx/tasks/domain.yml" + url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/{{ item }}.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/tasks/{{ item }}.yml" + with_items: + - domain + - ssl - name: Fetch nginx defaults. ansible.builtin.get_url: url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/defaults/main.yml - dest: "{{ _ce_deploy_base_dir }}/roles/nginx/defaults/main.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/defaults/main.yml" - name: Fetch nginx templates. ansible.builtin.get_url: url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/templates/{{ item }}" - dest: "{{ _ce_deploy_base_dir }}/roles/nginx/templates/{{ item }}" + dest: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/templates/{{ item }}" with_items: - vhosts.j2 - cloudwatch-vhost.json.j2 @@ -89,7 +92,7 @@ - name: Fetch ssl tasks. ansible.builtin.get_url: url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/ssl/tasks/{{ item }}.yml" - dest: "{{ _ce_deploy_base_dir }}/roles/ssl/tasks/{{ item }}.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/debian/ssl/tasks/{{ item }}.yml" with_items: - copy - generate @@ -102,12 +105,12 @@ - name: Fetch ssl defaults. ansible.builtin.get_url: url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/ssl/defaults/main.yml - dest: "{{ _ce_deploy_base_dir }}/roles/ssl/defaults/main.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/debian/ssl/defaults/main.yml" - name: Fetch ssl templates. ansible.builtin.get_url: url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/ssl/templates/{{ item }}" - dest: "{{ _ce_deploy_base_dir }}/roles/nginx/ssl/{{ item }}" + dest: "{{ _ce_deploy_base_dir }}/roles/debian/ssl/templates/{{ item }}" with_items: - le_cron.sh.j2 From 7a0f1266c0d4a9485c357321cb48a5a0e6309459 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 19 Jul 2024 14:10:48 +0200 Subject: [PATCH 142/190] Feature branching pr devel (#491) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. * Adding syncing for feature branches. (#346) * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/tasks/deploy.yml | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 312ae8b3..644688cf 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -68,13 +68,16 @@ - defaults - templates - - name: Fetch nginx tasks. + - name: Fetch nginx domain.yml. ansible.builtin.get_url: - url: "https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/{{ item }}.yml" - dest: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/tasks/{{ item }}.yml" - with_items: - - domain - - ssl + url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/domain.yml + dest: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/tasks/domain.yml" + + # Because of the way it is called, this needs putting into the playbook directory. + - name: Fetch nginx ssl.yml. + ansible.builtin.get_url: + url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/ssl.yml + dest: "{{ _ce_deploy_build_dir }}/deploy/ssl.yml" - name: Fetch nginx defaults. ansible.builtin.get_url: @@ -123,7 +126,7 @@ become: true block: - name: Generate domain specific configuration. - ansible.builtin.include_tasks: "{{ _ce_deploy_base_dir }}/roles/nginx/tasks/domain.yml" + ansible.builtin.include_tasks: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/tasks/domain.yml" with_items: "{{ deploy_code.feature_branch.domains }}" loop_control: loop_var: domain From 27de83ed91dea96118d889e7cf7ad9d3c658f755 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Fri, 19 Jul 2024 14:18:11 +0200 Subject: [PATCH 143/190] Feature branching pr devel (#492) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. * Adding syncing for feature branches. (#346) * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/tasks/deploy.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 644688cf..032e1577 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -71,7 +71,7 @@ - name: Fetch nginx domain.yml. ansible.builtin.get_url: url: https://raw.githubusercontent.com/codeenigma/ce-provision/2.x/roles/debian/nginx/tasks/domain.yml - dest: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/tasks/domain.yml" + dest: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/domain.yml" # Because of the way it is called, this needs putting into the playbook directory. - name: Fetch nginx ssl.yml. @@ -126,7 +126,7 @@ become: true block: - name: Generate domain specific configuration. - ansible.builtin.include_tasks: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/tasks/domain.yml" + ansible.builtin.include_tasks: "{{ _ce_deploy_base_dir }}/roles/debian/nginx/domain.yml" with_items: "{{ deploy_code.feature_branch.domains }}" loop_control: loop_var: domain From 63929fd0dd320ae58c863c9e34d3abf5146b43d3 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Mon, 5 Aug 2024 11:43:02 +0300 Subject: [PATCH 144/190] Additional symlink task fix for squashfs pr devel (#493) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- docs/roles/composer.md | 2 +- docs/roles/config_generate.md | 2 +- docs/roles/deploy_code.md | 2 +- docs/roles/npm.md | 2 +- roles/composer/README.md | 2 +- roles/composer/defaults/main.yml | 2 +- roles/config_generate/README.md | 2 +- roles/config_generate/defaults/main.yml | 2 +- roles/deploy_code/README.md | 2 +- roles/deploy_code/defaults/main.yml | 2 +- roles/deploy_code/tasks/deploy.yml | 15 ++++++++++++++- roles/live_symlink/defaults/main.yml | 2 +- roles/npm/README.md | 2 +- roles/npm/defaults/main.yml | 2 +- 14 files changed, 27 insertions(+), 14 deletions(-) diff --git a/docs/roles/composer.md b/docs/roles/composer.md index 74765170..c2fd2b1d 100644 --- a/docs/roles/composer.md +++ b/docs/roles/composer.md @@ -14,7 +14,7 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/docs/roles/config_generate.md b/docs/roles/config_generate.md index fe7aecfe..941b0680 100644 --- a/docs/roles/config_generate.md +++ b/docs/roles/config_generate.md @@ -11,7 +11,7 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/docs/roles/deploy_code.md b/docs/roles/deploy_code.md index 5d4cc722..fa1fec71 100644 --- a/docs/roles/deploy_code.md +++ b/docs/roles/deploy_code.md @@ -89,7 +89,7 @@ deploy_code: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/docs/roles/npm.md b/docs/roles/npm.md index 97f06cdc..45b401b0 100644 --- a/docs/roles/npm.md +++ b/docs/roles/npm.md @@ -18,7 +18,7 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/composer/README.md b/roles/composer/README.md index 74765170..c2fd2b1d 100644 --- a/roles/composer/README.md +++ b/roles/composer/README.md @@ -14,7 +14,7 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/composer/defaults/main.yml b/roles/composer/defaults/main.yml index d0dfbd1d..78876efc 100644 --- a/roles/composer/defaults/main.yml +++ b/roles/composer/defaults/main.yml @@ -7,7 +7,7 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/config_generate/README.md b/roles/config_generate/README.md index fe7aecfe..941b0680 100644 --- a/roles/config_generate/README.md +++ b/roles/config_generate/README.md @@ -11,7 +11,7 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/config_generate/defaults/main.yml b/roles/config_generate/defaults/main.yml index 8f1987eb..e9eb308a 100644 --- a/roles/config_generate/defaults/main.yml +++ b/roles/config_generate/defaults/main.yml @@ -3,7 +3,7 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/deploy_code/README.md b/roles/deploy_code/README.md index 5d4cc722..fa1fec71 100644 --- a/roles/deploy_code/README.md +++ b/roles/deploy_code/README.md @@ -89,7 +89,7 @@ deploy_code: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/deploy_code/defaults/main.yml b/roles/deploy_code/defaults/main.yml index b8220475..b28b479d 100644 --- a/roles/deploy_code/defaults/main.yml +++ b/roles/deploy_code/defaults/main.yml @@ -11,7 +11,7 @@ deploy_code: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 032e1577..307a480c 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -28,7 +28,7 @@ - deploy_code.templates | length - deploy_operation == 'deploy' -- name: Create additional symlinks. +- name: Create additional symlinks in build dir. ansible.builtin.file: src: "{{ link.src }}" dest: "{{ deploy_path }}/{{ link.dest }}" @@ -38,6 +38,19 @@ loop_var: link when: - deploy_code.symlinks | length + - deploy_code.mount_type != "squashfs" + +- name: Create additional symlinks in deploy dir. + ansible.builtin.file: + src: "{{ link.src }}" + dest: "{{ live_symlink_dest }}/{{ link.dest }}" + state: link + with_items: "{{ deploy_code.symlinks }}" + loop_control: + loop_var: link + when: + - deploy_code.symlinks | length + - deploy_code.mount_type == "squashfs" # Additional vhost handling for feature branch builds. diff --git a/roles/live_symlink/defaults/main.yml b/roles/live_symlink/defaults/main.yml index eca23ad6..0a1aac69 100644 --- a/roles/live_symlink/defaults/main.yml +++ b/roles/live_symlink/defaults/main.yml @@ -6,7 +6,7 @@ live_symlink: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/npm/README.md b/roles/npm/README.md index 97f06cdc..45b401b0 100644 --- a/roles/npm/README.md +++ b/roles/npm/README.md @@ -18,7 +18,7 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' diff --git a/roles/npm/defaults/main.yml b/roles/npm/defaults/main.yml index a29fcac7..57e45056 100644 --- a/roles/npm/defaults/main.yml +++ b/roles/npm/defaults/main.yml @@ -11,7 +11,7 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # create: whether to create the target if it does not exists. + # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # - src: '/var/simplesaml/etc' From ab4ef4def55a754a0bf354479295daa8b0adbb40 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Mon, 5 Aug 2024 12:01:46 +0300 Subject: [PATCH 145/190] Additional symlink task fix for squashfs pr devel (#495) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/defaults/main.yml | 4 +++- roles/deploy_code/tasks/deploy.yml | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/roles/deploy_code/defaults/main.yml b/roles/deploy_code/defaults/main.yml index b28b479d..62e50b00 100644 --- a/roles/deploy_code/defaults/main.yml +++ b/roles/deploy_code/defaults/main.yml @@ -11,11 +11,13 @@ deploy_code: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 307a480c..ef38f63a 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -45,6 +45,7 @@ src: "{{ link.src }}" dest: "{{ live_symlink_dest }}/{{ link.dest }}" state: link + force: "{{ link.force }}" with_items: "{{ deploy_code.symlinks }}" loop_control: loop_var: link From 51b4b0a856ef5ff26d326c0e37b91cac28c63933 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 6 Aug 2024 01:46:33 +0300 Subject: [PATCH 146/190] Additional symlink task fix for squashfs pr devel (#496) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/tasks/deploy.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index ef38f63a..68d40d4a 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -52,6 +52,7 @@ when: - deploy_code.symlinks | length - deploy_code.mount_type == "squashfs" + - deploy_operation == 'deploy' # Additional vhost handling for feature branch builds. From 9971a12738da23fa6232d6948f54d5247b610a68 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 6 Aug 2024 02:42:49 +0300 Subject: [PATCH 147/190] Additional symlink task fix for squashfs pr devel (#497) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- docs/roles/composer.md | 4 +++- docs/roles/config_generate.md | 4 +++- docs/roles/deploy_code.md | 4 +++- docs/roles/npm.md | 4 +++- roles/composer/README.md | 4 +++- roles/composer/defaults/main.yml | 4 +++- roles/composer/tasks/main.yml | 1 + roles/config_generate/README.md | 4 +++- roles/config_generate/defaults/main.yml | 4 +++- roles/config_generate/tasks/main.yml | 1 + roles/deploy_code/README.md | 4 +++- roles/deploy_code/tasks/deploy.yml | 4 +++- roles/live_symlink/defaults/main.yml | 4 +++- roles/live_symlink/tasks/main.yml | 2 +- roles/npm/README.md | 4 +++- roles/npm/defaults/main.yml | 4 +++- roles/npm/tasks/main.yml | 1 + 17 files changed, 43 insertions(+), 14 deletions(-) diff --git a/docs/roles/composer.md b/docs/roles/composer.md index c2fd2b1d..47e40b29 100644 --- a/docs/roles/composer.md +++ b/docs/roles/composer.md @@ -14,11 +14,13 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/docs/roles/config_generate.md b/docs/roles/config_generate.md index 941b0680..b6b46aba 100644 --- a/docs/roles/config_generate.md +++ b/docs/roles/config_generate.md @@ -11,11 +11,13 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/docs/roles/deploy_code.md b/docs/roles/deploy_code.md index fa1fec71..634dbbe5 100644 --- a/docs/roles/deploy_code.md +++ b/docs/roles/deploy_code.md @@ -89,11 +89,13 @@ deploy_code: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/docs/roles/npm.md b/docs/roles/npm.md index 45b401b0..5a17e577 100644 --- a/docs/roles/npm.md +++ b/docs/roles/npm.md @@ -18,11 +18,13 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/composer/README.md b/roles/composer/README.md index c2fd2b1d..47e40b29 100644 --- a/roles/composer/README.md +++ b/roles/composer/README.md @@ -14,11 +14,13 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/composer/defaults/main.yml b/roles/composer/defaults/main.yml index 78876efc..314f8bd0 100644 --- a/roles/composer/defaults/main.yml +++ b/roles/composer/defaults/main.yml @@ -7,11 +7,13 @@ composer: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/composer/tasks/main.yml b/roles/composer/tasks/main.yml index bba5e02a..5bf70693 100644 --- a/roles/composer/tasks/main.yml +++ b/roles/composer/tasks/main.yml @@ -25,6 +25,7 @@ src: "{{ link.src }}" dest: "{{ deploy_path }}/{{ link.dest }}" state: link + force: "{{ link.force | default(false) }}" with_items: "{{ composer.symlinks }}" loop_control: loop_var: link diff --git a/roles/config_generate/README.md b/roles/config_generate/README.md index 941b0680..b6b46aba 100644 --- a/roles/config_generate/README.md +++ b/roles/config_generate/README.md @@ -11,11 +11,13 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/config_generate/defaults/main.yml b/roles/config_generate/defaults/main.yml index e9eb308a..5b4f50cf 100644 --- a/roles/config_generate/defaults/main.yml +++ b/roles/config_generate/defaults/main.yml @@ -3,11 +3,13 @@ config_generate: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/config_generate/tasks/main.yml b/roles/config_generate/tasks/main.yml index adc35b06..dacf9615 100644 --- a/roles/config_generate/tasks/main.yml +++ b/roles/config_generate/tasks/main.yml @@ -20,6 +20,7 @@ src: "{{ link.src }}" dest: "{{ deploy_path }}/{{ link.dest }}" state: link + force: "{{ link.force | default(false) }}" with_items: "{{ config_generate.symlinks }}" loop_control: loop_var: link diff --git a/roles/deploy_code/README.md b/roles/deploy_code/README.md index fa1fec71..634dbbe5 100644 --- a/roles/deploy_code/README.md +++ b/roles/deploy_code/README.md @@ -89,11 +89,13 @@ deploy_code: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 68d40d4a..95315038 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -33,6 +33,7 @@ src: "{{ link.src }}" dest: "{{ deploy_path }}/{{ link.dest }}" state: link + force: "{{ link.force | default(false) }}" with_items: "{{ deploy_code.symlinks }}" loop_control: loop_var: link @@ -45,7 +46,8 @@ src: "{{ link.src }}" dest: "{{ live_symlink_dest }}/{{ link.dest }}" state: link - force: "{{ link.force }}" + force: "{{ link.force | default(false) }}" + follow: false with_items: "{{ deploy_code.symlinks }}" loop_control: loop_var: link diff --git a/roles/live_symlink/defaults/main.yml b/roles/live_symlink/defaults/main.yml index 0a1aac69..51618671 100644 --- a/roles/live_symlink/defaults/main.yml +++ b/roles/live_symlink/defaults/main.yml @@ -6,11 +6,13 @@ live_symlink: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/live_symlink/tasks/main.yml b/roles/live_symlink/tasks/main.yml index fa1d9e97..d85e4e7f 100644 --- a/roles/live_symlink/tasks/main.yml +++ b/roles/live_symlink/tasks/main.yml @@ -38,7 +38,7 @@ dest: "{{ deploy_path }}/{{ link.dest }}" state: link follow: false - force: true + force: "{{ link.force | default(true) }}" with_items: "{{ live_symlink.symlinks }}" loop_control: loop_var: link diff --git a/roles/npm/README.md b/roles/npm/README.md index 45b401b0..5a17e577 100644 --- a/roles/npm/README.md +++ b/roles/npm/README.md @@ -18,11 +18,13 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/npm/defaults/main.yml b/roles/npm/defaults/main.yml index 57e45056..49321d19 100644 --- a/roles/npm/defaults/main.yml +++ b/roles/npm/defaults/main.yml @@ -11,11 +11,13 @@ npm: # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') - # force: false # Set to true for the creation of the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. + # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: true # - src: '/var/simplesaml/etc' # dest: 'vendor/simplesamlphp/simplesamlphp/config' + # force: false symlinks: [] # Specify any additional templates to generate, with src (template) and dest (file). # src: name of a template, in the "templates" dir relative to your playbook. diff --git a/roles/npm/tasks/main.yml b/roles/npm/tasks/main.yml index f5372b0f..09487313 100644 --- a/roles/npm/tasks/main.yml +++ b/roles/npm/tasks/main.yml @@ -24,6 +24,7 @@ src: "{{ link.src }}" dest: "{{ deploy_path }}/{{ link.dest }}" state: link + force: "{{ link.force | default(false) }}" with_items: "{{ npm.symlinks }}" loop_control: loop_var: link From 71fe358735bf3b1a5a435a40535d98a6d4451f9d Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 6 Aug 2024 03:29:01 +0300 Subject: [PATCH 148/190] Additional symlink task fix for squashfs pr devel (#498) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/deploy_code/tasks/deploy.yml | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/roles/deploy_code/tasks/deploy.yml b/roles/deploy_code/tasks/deploy.yml index 95315038..a9d74d2e 100644 --- a/roles/deploy_code/tasks/deploy.yml +++ b/roles/deploy_code/tasks/deploy.yml @@ -39,22 +39,6 @@ loop_var: link when: - deploy_code.symlinks | length - - deploy_code.mount_type != "squashfs" - -- name: Create additional symlinks in deploy dir. - ansible.builtin.file: - src: "{{ link.src }}" - dest: "{{ live_symlink_dest }}/{{ link.dest }}" - state: link - force: "{{ link.force | default(false) }}" - follow: false - with_items: "{{ deploy_code.symlinks }}" - loop_control: - loop_var: link - when: - - deploy_code.symlinks | length - - deploy_code.mount_type == "squashfs" - - deploy_operation == 'deploy' # Additional vhost handling for feature branch builds. From d71047425148b02cbb157f45194faa602399a17c Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 20 Aug 2024 11:52:01 +0300 Subject: [PATCH 149/190] Fix cachetool url pr devel (#499) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/cli/cachetool/tasks/main.yml | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/roles/cli/cachetool/tasks/main.yml b/roles/cli/cachetool/tasks/main.yml index c0cbce61..c9855743 100644 --- a/roles/cli/cachetool/tasks/main.yml +++ b/roles/cli/cachetool/tasks/main.yml @@ -27,7 +27,7 @@ - name: Download latest cachetool installer if PHP is 8.1 or newer. ansible.builtin.get_url: - url: "http://gordalina.github.io/cachetool/downloads/cachetool.phar" + url: "https://gordalina.github.io/cachetool/downloads/cachetool.phar" dest: "{{ cachetool_bin }}" mode: 0755 when: @@ -35,7 +35,7 @@ - name: Download cachetool version 8.5.0 installer if PHP is 8.0. ansible.builtin.get_url: - url: "http://gordalina.github.io/cachetool/downloads/cachetool-8.5.0.phar" + url: "https://gordalina.github.io/cachetool/downloads/cachetool-8.5.0.phar" dest: "{{ cachetool_bin }}" mode: 0755 when: @@ -43,7 +43,7 @@ - name: Download cachetool version 7.1.0 installer if PHP is 7.3 or newer. ansible.builtin.get_url: - url: "http://gordalina.github.io/cachetool/downloads/cachetool-7.1.0.phar" + url: "https://gordalina.github.io/cachetool/downloads/cachetool-7.1.0.phar" dest: "{{ cachetool_bin }}" mode: 0755 when: @@ -52,7 +52,7 @@ - name: Download cachetool version 5.1.3 installer if PHP is 7.2. ansible.builtin.get_url: - url: "http://gordalina.github.io/cachetool/downloads/cachetool-5.1.3.phar" + url: "https://gordalina.github.io/cachetool/downloads/cachetool-5.1.3.phar" dest: "{{ cachetool_bin }}" mode: 0755 when: @@ -60,7 +60,7 @@ - name: Download cachetool version 4.1.1 installer if PHP is 7.1. ansible.builtin.get_url: - url: "http://gordalina.github.io/cachetool/downloads/cachetool-4.1.1.phar" + url: "https://gordalina.github.io/cachetool/downloads/cachetool-4.1.1.phar" dest: "{{ cachetool_bin }}" mode: 0755 when: @@ -68,7 +68,7 @@ - name: Download cachetool version 3.2.2 installer if PHP version is too old. ansible.builtin.get_url: - url: "http://gordalina.github.io/cachetool/downloads/cachetool-3.2.2.phar" + url: "https://gordalina.github.io/cachetool/downloads/cachetool-3.2.2.phar" dest: "{{ cachetool_bin }}" mode: 0755 when: @@ -76,7 +76,7 @@ - name: "Download the specified {{ cachetool.version }} cachetool version installer." ansible.builtin.get_url: - url: "http://gordalina.github.io/cachetool/downloads/cachetool-{{ cachetool.version }}.phar" + url: "https://gordalina.github.io/cachetool/downloads/cachetool-{{ cachetool.version }}.phar" dest: "{{ cachetool_bin }}" mode: 0755 when: From db1c5d95e3623b9f367c5b0d818adf90831e23de Mon Sep 17 00:00:00 2001 From: nfawbert <62660788+nfawbert@users.noreply.github.com> Date: Thu, 26 Sep 2024 10:51:51 +0100 Subject: [PATCH 150/190] R70771 strip incompatible sandbox syntax pr devel (#501) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden --- roles/sync/database_sync/database_sync-mysql/tasks/sync.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 0ae03be9..6b9d7dcd 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -184,6 +184,11 @@ ansible.builtin.command: cmd: "mysql --defaults-extra-file={{ database.target.credentials_file }} -e 'create database `{{ mysql_sync_target_database }}`;'" +- name: Remove lines containing "sandbox mode" from SQL dump. + ansible.builtin.shell: "sed -i '/sandbox mode/d' {{ mysql_sync_target_dump_unpacked_path }}" + args: + executable: /bin/bash + - name: Repopulate database from dump. ansible.builtin.shell: "mysql --defaults-extra-file={{ database.target.credentials_file }} {{ mysql_sync_target_database }} < {{ mysql_sync_target_dump_unpacked_path }}" args: From a9244072e8c5775e90140f89f6b967f0fbd03444 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Thu, 24 Oct 2024 16:46:46 +0300 Subject: [PATCH 151/190] R70963 aws profile pr devel (#503) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * r70963-aws-profile --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/_exit/tasks/main.yml | 2 +- roles/_init/tasks/main.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/_exit/tasks/main.yml b/roles/_exit/tasks/main.yml index 640f3a18..782ece63 100644 --- a/roles/_exit/tasks/main.yml +++ b/roles/_exit/tasks/main.yml @@ -2,7 +2,7 @@ # If we are operating on an AWS ASG then resume autoscaling. - name: Enable all autoscale processes on ASG. ansible.builtin.command: > - aws autoscaling resume-processes --auto-scaling-group-name {{ aws_asg.name }} --region {{ aws_asg.region }} + aws --profile {{ boto_profile }} autoscaling resume-processes --auto-scaling-group-name {{ aws_asg.name }} --region {{ aws_asg.region }} delegate_to: localhost when: - aws_asg.name is defined diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 6f0cd830..bb5b4573 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -158,7 +158,7 @@ # @TODO - the autoscaling_group module can do this - https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_module.html - name: Disable autoscale processes on ASG. ansible.builtin.command: > - aws autoscaling suspend-processes --auto-scaling-group-name {{ aws_asg.name }} --scaling-processes {{ aws_asg.suspend_processes }} --region {{ aws_asg.region }} + aws --profile {{ boto_profile }} autoscaling suspend-processes --auto-scaling-group-name {{ aws_asg.name }} --scaling-processes {{ aws_asg.suspend_processes }} --region {{ aws_asg.region }} delegate_to: localhost when: - aws_asg.name is defined From b2d39b7ce33a6d3bdb9742b220f845c9cb32e751 Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Tue, 29 Oct 2024 10:44:13 +0200 Subject: [PATCH 152/190] R70963 aws profile pr devel (#504) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * r70963-aws-profile * revert r70963-aws-profile --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/_exit/tasks/main.yml | 2 +- roles/_init/tasks/main.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/_exit/tasks/main.yml b/roles/_exit/tasks/main.yml index 782ece63..640f3a18 100644 --- a/roles/_exit/tasks/main.yml +++ b/roles/_exit/tasks/main.yml @@ -2,7 +2,7 @@ # If we are operating on an AWS ASG then resume autoscaling. - name: Enable all autoscale processes on ASG. ansible.builtin.command: > - aws --profile {{ boto_profile }} autoscaling resume-processes --auto-scaling-group-name {{ aws_asg.name }} --region {{ aws_asg.region }} + aws autoscaling resume-processes --auto-scaling-group-name {{ aws_asg.name }} --region {{ aws_asg.region }} delegate_to: localhost when: - aws_asg.name is defined diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index bb5b4573..6f0cd830 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -158,7 +158,7 @@ # @TODO - the autoscaling_group module can do this - https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_module.html - name: Disable autoscale processes on ASG. ansible.builtin.command: > - aws --profile {{ boto_profile }} autoscaling suspend-processes --auto-scaling-group-name {{ aws_asg.name }} --scaling-processes {{ aws_asg.suspend_processes }} --region {{ aws_asg.region }} + aws autoscaling suspend-processes --auto-scaling-group-name {{ aws_asg.name }} --scaling-processes {{ aws_asg.suspend_processes }} --region {{ aws_asg.region }} delegate_to: localhost when: - aws_asg.name is defined From f33d0bef467895abd8cc7089f1742f20013ca074 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Thu, 28 Nov 2024 17:08:10 +0100 Subject: [PATCH 153/190] Updating crons mailto pr devel (#505) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto * Forgot-variable-drupal8 --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/defaults/main.yml | 4 +++- roles/cron/cron_drupal7/tasks/job.yml | 1 + roles/cron/cron_drupal8/defaults/main.yml | 2 ++ roles/cron/cron_drupal8/tasks/job.yml | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) diff --git a/roles/cron/cron_drupal7/defaults/main.yml b/roles/cron/cron_drupal7/defaults/main.yml index 419e9c4b..4dc29105 100644 --- a/roles/cron/cron_drupal7/defaults/main.yml +++ b/roles/cron/cron_drupal7/defaults/main.yml @@ -12,8 +12,10 @@ drupal: # month: job: cron # disabled: true + mailto: "{{ drupal.cron_mailto | default('') }}" + cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false # If defer is set to true, the Ansible target must be declared with defer_target. If using a group, include the index. For example, _ce_www_dev[0] defer_target: "" - drush_location: "{{ drush_bin }}" # you might specify another location, e.g. "{{ deploy_path }}/vendor/bin/drush" \ No newline at end of file + drush_location: "{{ drush_bin }}" # you might specify another location, e.g. "{{ deploy_path }}/vendor/bin/drush" diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index b89a0b05..f2eb2ff7 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -31,4 +31,5 @@ job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" + mailto: "{{ entry.mailto if entry.mailto is defined else omit }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/defaults/main.yml b/roles/cron/cron_drupal8/defaults/main.yml index 9d4b90f6..4dc29105 100644 --- a/roles/cron/cron_drupal8/defaults/main.yml +++ b/roles/cron/cron_drupal8/defaults/main.yml @@ -12,6 +12,8 @@ drupal: # month: job: cron # disabled: true + mailto: "{{ drupal.cron_mailto | default('') }}" + cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false # If defer is set to true, the Ansible target must be declared with defer_target. If using a group, include the index. For example, _ce_www_dev[0] diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 7fa23c3e..77f3c6ae 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -37,4 +37,5 @@ job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" + mailto: "{{ entry.mailto if entry.mailto is defined else omit }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 067475dfb72ae024864c33d8441cfbe1e97d10a3 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Fri, 6 Dec 2024 09:10:45 +0100 Subject: [PATCH 154/190] Updating cron path fixing mailto pr devel (#507) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- docs/_Sidebar.md | 1 + docs/roles/cache_clear/cache_clear-symfony.md | 14 ++++++++++++++ docs/roles/cron/cron_drupal7.md | 3 +++ docs/roles/cron/cron_drupal8.md | 2 ++ docs/roles/deploy_code.md | 8 ++++++++ .../sync/database_sync/database_sync-mysql.md | 6 +++++- roles/cache_clear/cache_clear-symfony/README.md | 7 +++++++ roles/cron/cron_drupal7/README.md | 3 +++ roles/cron/cron_drupal7/defaults/main.yml | 2 +- roles/cron/cron_drupal7/tasks/job.yml | 4 ++-- roles/cron/cron_drupal8/README.md | 2 ++ roles/cron/cron_drupal8/defaults/main.yml | 2 +- roles/cron/cron_drupal8/tasks/job.yml | 9 ++------- roles/deploy_code/README.md | 8 ++++++++ .../database_sync/database_sync-mysql/README.md | 6 +++++- 15 files changed, 64 insertions(+), 13 deletions(-) create mode 100644 docs/roles/cache_clear/cache_clear-symfony.md diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index b8c4d52e..b1f28b08 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -10,6 +10,7 @@ - [Matomo](/roles/cache_clear/cache_clear-matomo) - [Drupal 8](/roles/cache_clear/cache_clear-mautic) - [Opcache](/roles/cache_clear/cache_clear-opcache) + - [Symfony](/roles/cache_clear/cache_clear-symfony) - [ce-dev](/roles/ce_dev) - [ce-dev prebuild](/roles/ce_dev/ce_dev_prebuild) - [CLI Tools](/roles/cli) diff --git a/docs/roles/cache_clear/cache_clear-symfony.md b/docs/roles/cache_clear/cache_clear-symfony.md new file mode 100644 index 00000000..65c608f9 --- /dev/null +++ b/docs/roles/cache_clear/cache_clear-symfony.md @@ -0,0 +1,14 @@ +# Symfony +Clears the Symfony application cache. + + + + +## Default variables +```yaml +--- +cache_clear_symfony_env: prod + +``` + + diff --git a/docs/roles/cron/cron_drupal7.md b/docs/roles/cron/cron_drupal7.md index 14356d15..dd46727a 100644 --- a/docs/roles/cron/cron_drupal7.md +++ b/docs/roles/cron/cron_drupal7.md @@ -19,11 +19,14 @@ drupal: # month: job: cron # disabled: true + # mailto: "{{ drupal.cron_mailto | default('') }}" # Each cron can have it's own mailto and can be configured to use different e-mail addresses. + cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false # If defer is set to true, the Ansible target must be declared with defer_target. If using a group, include the index. For example, _ce_www_dev[0] defer_target: "" drush_location: "{{ drush_bin }}" # you might specify another location, e.g. "{{ deploy_path }}/vendor/bin/drush" + ``` diff --git a/docs/roles/cron/cron_drupal8.md b/docs/roles/cron/cron_drupal8.md index bafd49b0..7ff5f24c 100644 --- a/docs/roles/cron/cron_drupal8.md +++ b/docs/roles/cron/cron_drupal8.md @@ -19,6 +19,8 @@ drupal: # month: job: cron # disabled: true + # mailto: "{{ drupal.cron_mailto | default('') }}" # Each cron can have it's own mailto and can be configured to use different e-mail addresses. + cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false # If defer is set to true, the Ansible target must be declared with defer_target. If using a group, include the index. For example, _ce_www_dev[0] diff --git a/docs/roles/deploy_code.md b/docs/roles/deploy_code.md index 634dbbe5..b3860144 100644 --- a/docs/roles/deploy_code.md +++ b/docs/roles/deploy_code.md @@ -85,7 +85,15 @@ Deploying code with autoscaling clusters relies on [cloud-init](https://cloudini ## Default variables ```yaml --- +# Required for NGINX config in feature branching. +_ce_provision_data_dir: /home/deploy/ce-deploy/data + deploy_code: + # Feature branching config. + feature_branch: + enabled: false + # NGINX domains to build, see ce-provision for docs: https://github.com/codeenigma/ce-provision/blob/2.x/roles/debian/nginx/defaults/main.yml#L133 + domains: [] # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') diff --git a/docs/roles/sync/database_sync/database_sync-mysql.md b/docs/roles/sync/database_sync/database_sync-mysql.md index ab07346d..1fd2afe8 100644 --- a/docs/roles/sync/database_sync/database_sync-mysql.md +++ b/docs/roles/sync/database_sync/database_sync-mysql.md @@ -25,8 +25,9 @@ mysql_sync: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" # This can be of types: # - rolling: (database backups). In that case we'll need build parameters.@todo - # - fixed: "fixed" database name + # - fixed: "fixed" database name # currently this var means the same as "dump" # - dump: Use an existing dump. In that case, the "database" variable is the absolute file path. + # This parameter is ignored if var 'path_on_deploy_server' is defined. type: fixed # For "rolling builds", so we can compute the database name. build_id: mybuildprod @@ -37,6 +38,9 @@ mysql_sync: dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/regular" # If the source is on an ASG, provide the ASG name here. Otherwise, leave empty. asg: "" + # Uncomment and specify full path if you want to use your own dump file on the deploy server (gzip compressed). + # In this case it's the only variable you need in 'database.source'. + #path_on_deploy_server: "/home/{{ deploy_user }}/compressed_database_dump.gz" target: database: "{{ project_name }}_dev" credentials_file: "/home/{{ deploy_user }}/.mysql.creds" diff --git a/roles/cache_clear/cache_clear-symfony/README.md b/roles/cache_clear/cache_clear-symfony/README.md index 8bb3fde2..65c608f9 100644 --- a/roles/cache_clear/cache_clear-symfony/README.md +++ b/roles/cache_clear/cache_clear-symfony/README.md @@ -4,4 +4,11 @@ Clears the Symfony application cache. +## Default variables +```yaml +--- +cache_clear_symfony_env: prod + +``` + diff --git a/roles/cron/cron_drupal7/README.md b/roles/cron/cron_drupal7/README.md index 14356d15..dd46727a 100644 --- a/roles/cron/cron_drupal7/README.md +++ b/roles/cron/cron_drupal7/README.md @@ -19,11 +19,14 @@ drupal: # month: job: cron # disabled: true + # mailto: "{{ drupal.cron_mailto | default('') }}" # Each cron can have it's own mailto and can be configured to use different e-mail addresses. + cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false # If defer is set to true, the Ansible target must be declared with defer_target. If using a group, include the index. For example, _ce_www_dev[0] defer_target: "" drush_location: "{{ drush_bin }}" # you might specify another location, e.g. "{{ deploy_path }}/vendor/bin/drush" + ``` diff --git a/roles/cron/cron_drupal7/defaults/main.yml b/roles/cron/cron_drupal7/defaults/main.yml index 4dc29105..636557ca 100644 --- a/roles/cron/cron_drupal7/defaults/main.yml +++ b/roles/cron/cron_drupal7/defaults/main.yml @@ -12,7 +12,7 @@ drupal: # month: job: cron # disabled: true - mailto: "{{ drupal.cron_mailto | default('') }}" + # mailto: "{{ drupal.cron_mailto | default('') }}" # Each cron can have it's own mailto and can be configured to use different e-mail addresses. cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index f2eb2ff7..ee1a136c 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -1,7 +1,7 @@ --- - name: Define cron job command. ansible.builtin.set_fact: - _cron_job_command: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }}" + _cron_job_command: "cd {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }} > /dev/null 2>&1" - name: Define cron job command if deferred (ASG). ansible.builtin.set_fact: @@ -31,5 +31,5 @@ job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" - mailto: "{{ entry.mailto if entry.mailto is defined else omit }}" + mailto: "{{ entry.mailto | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/README.md b/roles/cron/cron_drupal8/README.md index bafd49b0..7ff5f24c 100644 --- a/roles/cron/cron_drupal8/README.md +++ b/roles/cron/cron_drupal8/README.md @@ -19,6 +19,8 @@ drupal: # month: job: cron # disabled: true + # mailto: "{{ drupal.cron_mailto | default('') }}" # Each cron can have it's own mailto and can be configured to use different e-mail addresses. + cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false # If defer is set to true, the Ansible target must be declared with defer_target. If using a group, include the index. For example, _ce_www_dev[0] diff --git a/roles/cron/cron_drupal8/defaults/main.yml b/roles/cron/cron_drupal8/defaults/main.yml index 4dc29105..636557ca 100644 --- a/roles/cron/cron_drupal8/defaults/main.yml +++ b/roles/cron/cron_drupal8/defaults/main.yml @@ -12,7 +12,7 @@ drupal: # month: job: cron # disabled: true - mailto: "{{ drupal.cron_mailto | default('') }}" + # mailto: "{{ drupal.cron_mailto | default('') }}" # Each cron can have it's own mailto and can be configured to use different e-mail addresses. cron_mailto: "" # If the sites are being deployed to an ASG, setting defer to true will create the crontab entry on the deploy server rather than all of the app servers. defer: false diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 77f3c6ae..e58a2d92 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -1,14 +1,9 @@ --- - name: Define cron job command. ansible.builtin.set_fact: - _cron_job_command: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }}" + _cron_job_command: "cd {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }} > /dev/null 2>&1" when: deploy_operation == "deploy" -- name: Revert cron job command. - ansible.builtin.set_fact: - _cron_job_command: "cd {{ previous_deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }}" - when: deploy_operation == "revert" - - name: Define cron job command if deferred (ASG). ansible.builtin.set_fact: _cron_job_command: "cd {{ _ce_deploy_base_dir }} && {{ _ce_deploy_ansible_location }} {{ drupal.defer_target }} -m shell -a \"{{ _cron_job_command }}\"" @@ -37,5 +32,5 @@ job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" - mailto: "{{ entry.mailto if entry.mailto is defined else omit }}" + mailto: "{{ entry.mailto | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/deploy_code/README.md b/roles/deploy_code/README.md index 634dbbe5..b3860144 100644 --- a/roles/deploy_code/README.md +++ b/roles/deploy_code/README.md @@ -85,7 +85,15 @@ Deploying code with autoscaling clusters relies on [cloud-init](https://cloudini ## Default variables ```yaml --- +# Required for NGINX config in feature branching. +_ce_provision_data_dir: /home/deploy/ce-deploy/data + deploy_code: + # Feature branching config. + feature_branch: + enabled: false + # NGINX domains to build, see ce-provision for docs: https://github.com/codeenigma/ce-provision/blob/2.x/roles/debian/nginx/defaults/main.yml#L133 + domains: [] # Specify any additional symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') diff --git a/roles/sync/database_sync/database_sync-mysql/README.md b/roles/sync/database_sync/database_sync-mysql/README.md index ab07346d..1fd2afe8 100644 --- a/roles/sync/database_sync/database_sync-mysql/README.md +++ b/roles/sync/database_sync/database_sync-mysql/README.md @@ -25,8 +25,9 @@ mysql_sync: credentials_file: "/home/{{ deploy_user }}/.mysql.creds" # This can be of types: # - rolling: (database backups). In that case we'll need build parameters.@todo - # - fixed: "fixed" database name + # - fixed: "fixed" database name # currently this var means the same as "dump" # - dump: Use an existing dump. In that case, the "database" variable is the absolute file path. + # This parameter is ignored if var 'path_on_deploy_server' is defined. type: fixed # For "rolling builds", so we can compute the database name. build_id: mybuildprod @@ -37,6 +38,9 @@ mysql_sync: dumps_directory: "/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/db_backups/mysql/regular" # If the source is on an ASG, provide the ASG name here. Otherwise, leave empty. asg: "" + # Uncomment and specify full path if you want to use your own dump file on the deploy server (gzip compressed). + # In this case it's the only variable you need in 'database.source'. + #path_on_deploy_server: "/home/{{ deploy_user }}/compressed_database_dump.gz" target: database: "{{ project_name }}_dev" credentials_file: "/home/{{ deploy_user }}/.mysql.creds" From fd3d4f253911d09e1603bb15c45166b1b1292166 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 09:50:39 +0100 Subject: [PATCH 155/190] Updating cron path fixing mailto pr devel (#509) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 15 ++++++++++++++- roles/cron/cron_drupal8/tasks/job.yml | 15 ++++++++++++++- 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index ee1a136c..df0722d2 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -20,6 +20,20 @@ - drupal.defer is defined - drupal.defer +- name: Set global MAILTO for cron jobs (if defined) + community.general.cronvar: + name: MAILTO + value: "{{ drupal.cron_mailto | default(omit) }}" + state: present + when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 + +- name: Setup specific cron job with custom MAILTO + community.general.cronvar: + name: MAILTO + value: "{{ entry.mailto | default(drupal.cron_mailto | default(omit)) }}" + state: present + when: entry.mailto is defined + - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -31,5 +45,4 @@ job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" - mailto: "{{ entry.mailto | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index e58a2d92..71eb0d9f 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -21,6 +21,20 @@ - drupal.defer is defined - drupal.defer +- name: Set global MAILTO for cron jobs (if defined) + community.general.cronvar: + name: MAILTO + value: "{{ drupal.cron_mailto | default(omit) }}" + state: present + when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 + +- name: Setup specific cron job with custom MAILTO + community.general.cronvar: + name: MAILTO + value: "{{ entry.mailto | default(drupal.cron_mailto | default(omit)) }}" + state: present + when: entry.mailto is defined + - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -32,5 +46,4 @@ job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" - mailto: "{{ entry.mailto | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From ca815318939405c905242a93eea460a4a04b21ef Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 10:04:09 +0100 Subject: [PATCH 156/190] Updating cron path fixing mailto pr devel (#510) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 5 ++++- roles/cron/cron_drupal8/tasks/job.yml | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index df0722d2..ba6109be 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -42,7 +42,10 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: "{{ _cron_job_command }}" + job: | + {% if entry.mailto is defined %} + MAILTO={{ entry.mailto }} + {% endif %} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 71eb0d9f..d1e41df6 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -43,7 +43,10 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: "{{ _cron_job_command }}" + job: | + {% if entry.mailto is defined %} + MAILTO={{ entry.mailto }} + {% endif %} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 670f3243c454c779b1833e3b9e87bb5f134af801 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 10:21:22 +0100 Subject: [PATCH 157/190] Updating cron path fixing mailto pr devel (#511) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal8/tasks/job.yml | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index d1e41df6..0b4944e3 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -28,12 +28,14 @@ state: present when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 -- name: Setup specific cron job with custom MAILTO +- name: Setup specific MAILTO for cron jobs. community.general.cronvar: name: MAILTO value: "{{ entry.mailto | default(drupal.cron_mailto | default(omit)) }}" state: present - when: entry.mailto is defined + when: + - entry.mailto is defined + - entry.mailto | length > 0 - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: @@ -43,10 +45,7 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: | - {% if entry.mailto is defined %} - MAILTO={{ entry.mailto }} - {% endif %} + job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From a037f88aba315b18ce1ca7eadd3c5b138bd6a317 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 10:51:20 +0100 Subject: [PATCH 158/190] Updating cron path fixing mailto pr devel (#512) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 15 ++++----------- roles/cron/cron_drupal8/tasks/job.yml | 12 +++--------- 2 files changed, 7 insertions(+), 20 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index ba6109be..f8e8d608 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -27,13 +27,6 @@ state: present when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 -- name: Setup specific cron job with custom MAILTO - community.general.cronvar: - name: MAILTO - value: "{{ entry.mailto | default(drupal.cron_mailto | default(omit)) }}" - state: present - when: entry.mailto is defined - - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -42,10 +35,10 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: | - {% if entry.mailto is defined %} - MAILTO={{ entry.mailto }} - {% endif %} + env: + name: MAILTO + value: "{{ entry.mailto | default(drupal.cron_mailto | default('')) }}" + job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 0b4944e3..fdfd4f05 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -28,15 +28,6 @@ state: present when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 -- name: Setup specific MAILTO for cron jobs. - community.general.cronvar: - name: MAILTO - value: "{{ entry.mailto | default(drupal.cron_mailto | default(omit)) }}" - state: present - when: - - entry.mailto is defined - - entry.mailto | length > 0 - - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -45,6 +36,9 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" + env: + name: MAILTO + value: "{{ entry.mailto | default(drupal.cron_mailto | default('')) }}" job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" From 327f9af79eb7dbe6be6636382f3e5e8495d3bf06 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 11:11:09 +0100 Subject: [PATCH 159/190] Updating cron path fixing mailto pr devel (#513) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 18 ++++++++++++++---- roles/cron/cron_drupal8/tasks/job.yml | 18 ++++++++++++++---- 2 files changed, 28 insertions(+), 8 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index f8e8d608..d72306f6 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -27,6 +27,13 @@ state: present when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 +- name: Setup specific MAILTO for cron jobs if defined + community.general.cronvar: + name: MAILTO + value: "{{ entry.mailto | default(omit) }}" + state: present + when: entry.mailto is defined and entry.mailto | length > 0 + - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -35,10 +42,13 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - env: - name: MAILTO - value: "{{ entry.mailto | default(drupal.cron_mailto | default('')) }}" - job: "{{ _cron_job_command }}" + job: | + {% if entry.mailto is defined and entry.mailto | length > 0 %} + MAILTO={{ entry.mailto }} + {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} + MAILTO={{ drupal.cron_mailto }} + {% endif %} + {{ cronjob_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index fdfd4f05..59a436f5 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -28,6 +28,13 @@ state: present when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 +- name: Setup specific MAILTO for cron jobs if defined + community.general.cronvar: + name: MAILTO + value: "{{ entry.mailto | default(omit) }}" + state: present + when: entry.mailto is defined and entry.mailto | length > 0 + - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -36,10 +43,13 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - env: - name: MAILTO - value: "{{ entry.mailto | default(drupal.cron_mailto | default('')) }}" - job: "{{ _cron_job_command }}" + job: | + {% if entry.mailto is defined and entry.mailto | length > 0 %} + MAILTO={{ entry.mailto }} + {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} + MAILTO={{ drupal.cron_mailto }} + {% endif %} + {{ cronjob_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 2e64ea6bb3dd151f555985c7223c6914a7f73c57 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 11:16:38 +0100 Subject: [PATCH 160/190] Updating cron path fixing mailto pr devel (#514) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 2 +- roles/cron/cron_drupal8/tasks/job.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index d72306f6..d2a2c4bc 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -48,7 +48,7 @@ {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} MAILTO={{ drupal.cron_mailto }} {% endif %} - {{ cronjob_command }} + {{ _cronjob_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 59a436f5..69131e58 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -49,7 +49,7 @@ {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} MAILTO={{ drupal.cron_mailto }} {% endif %} - {{ cronjob_command }} + {{ _cronjob_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 62412da15ca92c94bff812e18cafe1ae8868f151 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 11:24:12 +0100 Subject: [PATCH 161/190] Updating cron path fixing mailto pr devel (#515) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 2 +- roles/cron/cron_drupal8/tasks/job.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index d2a2c4bc..cd96032f 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -48,7 +48,7 @@ {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} MAILTO={{ drupal.cron_mailto }} {% endif %} - {{ _cronjob_command }} + {{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 69131e58..0d996748 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -49,7 +49,7 @@ {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} MAILTO={{ drupal.cron_mailto }} {% endif %} - {{ _cronjob_command }} + {{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 5bc868bb9bb783c58e48905663c248067c028833 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Tue, 10 Dec 2024 11:47:27 +0100 Subject: [PATCH 162/190] Updating cron path fixing mailto pr devel (#516) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 8 +------- roles/cron/cron_drupal8/tasks/job.yml | 8 +------- 2 files changed, 2 insertions(+), 14 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index cd96032f..c52d67ce 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -42,13 +42,7 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: | - {% if entry.mailto is defined and entry.mailto | length > 0 %} - MAILTO={{ entry.mailto }} - {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} - MAILTO={{ drupal.cron_mailto }} - {% endif %} - {{ _cron_job_command }} + job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 0d996748..308d77bb 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -43,13 +43,7 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: | - {% if entry.mailto is defined and entry.mailto | length > 0 %} - MAILTO={{ entry.mailto }} - {% elif drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 %} - MAILTO={{ drupal.cron_mailto }} - {% endif %} - {{ _cron_job_command }} + job: "{{ _cron_job_command }}" state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From ace1c0b69d8ab64797bd1894305008ce19520b26 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 11 Dec 2024 10:34:28 +0100 Subject: [PATCH 163/190] Updating cron path fixing mailto pr devel (#517) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 11 +++-------- roles/cron/cron_drupal8/tasks/job.yml | 11 +++-------- 2 files changed, 6 insertions(+), 16 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index c52d67ce..f20388e7 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -27,13 +27,6 @@ state: present when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 -- name: Setup specific MAILTO for cron jobs if defined - community.general.cronvar: - name: MAILTO - value: "{{ entry.mailto | default(omit) }}" - state: present - when: entry.mailto is defined and entry.mailto | length > 0 - - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -42,7 +35,9 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: "{{ _cron_job_command }}" + job: > + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %} + {{ cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 308d77bb..9c074d89 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -28,13 +28,6 @@ state: present when: drupal.cron_mailto is defined and drupal.cron_mailto | length > 0 -- name: Setup specific MAILTO for cron jobs if defined - community.general.cronvar: - name: MAILTO - value: "{{ entry.mailto | default(omit) }}" - state: present - when: entry.mailto is defined and entry.mailto | length > 0 - - name: Setup Drupal cron tasks on app server. ansible.builtin.cron: name: "{{ project_name }}_{{ site.folder }}_{{ build_type }}_{{ entry.job }}" @@ -43,7 +36,9 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: "{{ _cron_job_command }}" + job: > + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %} + {{ cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From bf2b2d852eb6eda4907a1d584417934c785c6179 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:06:53 +0100 Subject: [PATCH 164/190] Updating cron path fixing mailto pr devel (#518) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 3 +-- roles/cron/cron_drupal8/tasks/job.yml | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index f20388e7..2d301991 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -36,8 +36,7 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %} - {{ cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 9c074d89..4b743469 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -37,8 +37,7 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %} - {{ cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 5c7f9dff30509b1bdad4b64ef05bda8571a08d93 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:29:08 +0100 Subject: [PATCH 165/190] Updating cron path fixing mailto pr devel (#519) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 2 +- roles/cron/cron_drupal8/tasks/job.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index 2d301991..307c9342 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -36,7 +36,7 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 4b743469..e3eeafeb 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -37,7 +37,7 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 9bebc0d5f8c68ba26e697f6edacf7d9adfa6feb9 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 18 Dec 2024 09:37:57 +0100 Subject: [PATCH 166/190] Updating cron path fixing mailto pr devel (#520) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 3 ++- roles/cron/cron_drupal8/tasks/job.yml | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index 307c9342..5db27375 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -36,7 +36,8 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ _cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} + {% endif %}{{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index e3eeafeb..2bd74273 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -37,7 +37,8 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} {% endif %}{{ _cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} + {% endif %}{{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From e2f9c124161d2bda4c81ae458450e9395993b13a Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 18 Dec 2024 10:29:46 +0100 Subject: [PATCH 167/190] Updating cron path fixing mailto pr devel (#521) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 8 +++++--- roles/cron/cron_drupal8/tasks/job.yml | 8 +++++--- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index 5db27375..289b4e87 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -1,7 +1,7 @@ --- - name: Define cron job command. ansible.builtin.set_fact: - _cron_job_command: "cd {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }} > /dev/null 2>&1" + _cron_job_command: "cd {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }} >/dev/null" - name: Define cron job command if deferred (ASG). ansible.builtin.set_fact: @@ -36,8 +36,10 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} - {% endif %}{{ _cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %} + MAILTO={{ entry.mailto }} + {% endif %} + { _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 2bd74273..b138b71b 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -1,7 +1,7 @@ --- - name: Define cron job command. ansible.builtin.set_fact: - _cron_job_command: "cd {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }} > /dev/null 2>&1" + _cron_job_command: "cd {{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }} && {{ drupal.drush_location }} {{ entry.job }} >/dev/null" when: deploy_operation == "deploy" - name: Define cron job command if deferred (ASG). @@ -37,8 +37,10 @@ weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" job: > - {% if entry.mailto is defined and entry.mailto | length > 0 %}MAILTO={{ entry.mailto }} - {% endif %}{{ _cron_job_command }} + {% if entry.mailto is defined and entry.mailto | length > 0 %} + MAILTO={{ entry.mailto }} + {% endif %} + { _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 884f495179abfbfae5af2998a73eaa1d6b5001c9 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 18 Dec 2024 10:44:18 +0100 Subject: [PATCH 168/190] Updating cron path fixing mailto pr devel (#522) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 2 +- roles/cron/cron_drupal8/tasks/job.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index 289b4e87..43e5e563 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -39,7 +39,7 @@ {% if entry.mailto is defined and entry.mailto | length > 0 %} MAILTO={{ entry.mailto }} {% endif %} - { _cron_job_command }} + {{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index b138b71b..29bdc272 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -40,7 +40,7 @@ {% if entry.mailto is defined and entry.mailto | length > 0 %} MAILTO={{ entry.mailto }} {% endif %} - { _cron_job_command }} + {{ _cron_job_command }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 16f225c46a268205955b051992018a97b091f86c Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 18 Dec 2024 10:57:32 +0100 Subject: [PATCH 169/190] Updating cron path fixing mailto pr devel (#523) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 2 +- roles/cron/cron_drupal8/tasks/job.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index 43e5e563..33f99e7c 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -35,7 +35,7 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: > + job: | {% if entry.mailto is defined and entry.mailto | length > 0 %} MAILTO={{ entry.mailto }} {% endif %} diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index 29bdc272..aea0826e 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -36,7 +36,7 @@ day: "{{ entry.day | default(omit) }}" weekday: "{{ entry.weekday | default(omit) }}" month: "{{ entry.month | default(omit) }}" - job: > + job: | {% if entry.mailto is defined and entry.mailto | length > 0 %} MAILTO={{ entry.mailto }} {% endif %} From 7ffc355009f06de1a23f333edec422e66ee4b5a7 Mon Sep 17 00:00:00 2001 From: drazenCE <140631110+drazenCE@users.noreply.github.com> Date: Wed, 18 Dec 2024 12:02:07 +0100 Subject: [PATCH 170/190] Updating cron path fixing mailto pr devel (#524) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> --- roles/cron/cron_drupal7/tasks/job.yml | 4 ++-- roles/cron/cron_drupal8/tasks/job.yml | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/roles/cron/cron_drupal7/tasks/job.yml b/roles/cron/cron_drupal7/tasks/job.yml index 33f99e7c..50194883 100644 --- a/roles/cron/cron_drupal7/tasks/job.yml +++ b/roles/cron/cron_drupal7/tasks/job.yml @@ -37,9 +37,9 @@ month: "{{ entry.month | default(omit) }}" job: | {% if entry.mailto is defined and entry.mailto | length > 0 %} - MAILTO={{ entry.mailto }} + MAILTO={{ entry.mailto | trim }} {% endif %} - {{ _cron_job_command }} + {{ _cron_job_command | trim }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" diff --git a/roles/cron/cron_drupal8/tasks/job.yml b/roles/cron/cron_drupal8/tasks/job.yml index aea0826e..23c7bd5b 100644 --- a/roles/cron/cron_drupal8/tasks/job.yml +++ b/roles/cron/cron_drupal8/tasks/job.yml @@ -38,9 +38,9 @@ month: "{{ entry.month | default(omit) }}" job: | {% if entry.mailto is defined and entry.mailto | length > 0 %} - MAILTO={{ entry.mailto }} + MAILTO={{ entry.mailto | trim }} {% endif %} - {{ _cron_job_command }} + {{ _cron_job_command | trim }} state: present disabled: "{{ entry.disabled | default(omit) }}" delegate_to: "{{ 'localhost' if drupal.defer else inventory_hostname }}" From 505fce9000cd4c1ac60787b0c0a9c89bf03ad0eb Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 30 Dec 2024 11:53:04 +0100 Subject: [PATCH 171/190] Mautic updates pr devel (#526) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- roles/_meta/deploy-mautic/tasks/main.yml | 30 ++++++++++----------- roles/cron/cron_mautic/tasks/main.yml | 7 +++-- roles/live_symlink/defaults/main.yml | 4 +-- roles/live_symlink/tasks/main.yml | 33 ++++++++++++++++-------- 4 files changed, 44 insertions(+), 30 deletions(-) diff --git a/roles/_meta/deploy-mautic/tasks/main.yml b/roles/_meta/deploy-mautic/tasks/main.yml index ab4a38d2..4ec69f31 100644 --- a/roles/_meta/deploy-mautic/tasks/main.yml +++ b/roles/_meta/deploy-mautic/tasks/main.yml @@ -1,37 +1,37 @@ --- # Default Mautic role. -- import_role: +- ansible.builtin.import_role: name: _init -- import_role: +- ansible.builtin.import_role: name: deploy_code -# - import_role: -# name: composer -# - import_role: +- ansible.builtin.import_role: + name: composer +# - ansible.builtin.import_role: # name: maintenance_mode # vars: # maintenance_mode: # operation: offline -- import_role: +- ansible.builtin.import_role: name: database_backup -- import_role: +- ansible.builtin.import_role: name: config_generate -- import_role: +- ansible.builtin.import_role: name: database_apply -- import_role: +- ansible.builtin.import_role: name: live_symlink -- import_role: +- ansible.builtin.import_role: name: cache_clear/cache_clear-opcache -- import_role: +- ansible.builtin.import_role: name: cache_clear/cache_clear-mautic -# - import_role: +# - ansible.builtin.import_role: # name: maintenance_mode # vars: # maintenance_mode: # operation: online -- import_role: +- ansible.builtin.import_role: name: cron/cron_database_backup -- import_role: +- ansible.builtin.import_role: name: cron/cron_mautic -- import_role: +- ansible.builtin.import_role: name: _exit diff --git a/roles/cron/cron_mautic/tasks/main.yml b/roles/cron/cron_mautic/tasks/main.yml index f0b6071b..403536dd 100644 --- a/roles/cron/cron_mautic/tasks/main.yml +++ b/roles/cron/cron_mautic/tasks/main.yml @@ -1,12 +1,15 @@ --- +- name: Set console path. + ansible.builtin.set_fact: + symfony_console_bin: "{{ symfony_console_bin | default('env php ' + deploy_path + '/bin/console') }}" - name: Setup Mautic cron tasks. cron: name: "{{ project_name }}_{{ build_type }}_{{ entry.job }}" minute: "{{ entry.minute }}" hour: "{{ entry.hour | default(omit) }}" - job: "env php {{ live_symlink_dest }}/{{ webroot }}/app/console {{ entry.job }}" + job: "env php {{ symfony_console_bin }} {{ entry.job }}" with_items: "{{ cron_mautic.entries }}" loop_control: loop_var: entry - when: deploy_operation == 'deploy' \ No newline at end of file + when: deploy_operation == 'deploy' diff --git a/roles/live_symlink/defaults/main.yml b/roles/live_symlink/defaults/main.yml index 51618671..8bf22492 100644 --- a/roles/live_symlink/defaults/main.yml +++ b/roles/live_symlink/defaults/main.yml @@ -3,11 +3,11 @@ deploy_user: 'deploy' live_symlink: # Either update (for successful builds) or revert. operation: update - # Specify any additional symlink to create, with src (target) and dest (link). + # Specify any additional directory symlink to create, with src (target) and dest (link). # src: can be either absolute or relative to the dest (eg. '/var/my_data', '/home/deploy/simplesaml', '../../../myconfig') # dest: can only be relative to the root of your repository (eg. 'www/themes/myassets', 'var/cache') # force: set to true to create the symlinks in two cases: the source file does not exist but will appear later; the destination exists and is a file. - # - src: '/home/{{ deploy_user }}//{{ project_name }}_{{ build_type }}/simplesaml' + # - src: '/home/{{ deploy_user }}/shared/{{ project_name }}_{{ build_type }}/simplesaml' # dest: 'vendor/simplesamlphp/simplesamlphp/config' # force: true # - src: '/var/simplesaml/etc' diff --git a/roles/live_symlink/tasks/main.yml b/roles/live_symlink/tasks/main.yml index d85e4e7f..63225e16 100644 --- a/roles/live_symlink/tasks/main.yml +++ b/roles/live_symlink/tasks/main.yml @@ -32,16 +32,27 @@ - live_symlink.templates | length - deploy_operation == 'deploy' -- name: Create additional symlinks. - ansible.builtin.file: - src: "{{ link.src }}" - dest: "{{ deploy_path }}/{{ link.dest }}" - state: link - follow: false - force: "{{ link.force | default(true) }}" - with_items: "{{ live_symlink.symlinks }}" - loop_control: - loop_var: link +- name: Handle additional symlinks. when: - live_symlink.symlinks | length - - deploy_operation == 'deploy' \ No newline at end of file + - deploy_operation == 'deploy' + block: + - name: Ensure additional symlink sources exist. + ansible.builtin.file: + path: "{{ link.src }}" + state: directory + mode: '0755' + with_items: "{{ live_symlink.symlinks }}" + loop_control: + loop_var: link + + - name: Create additional symlinks. + ansible.builtin.file: + src: "{{ link.src }}" + dest: "{{ deploy_path }}/{{ link.dest }}" + state: link + follow: false + force: "{{ link.force | default(true) }}" + with_items: "{{ live_symlink.symlinks }}" + loop_control: + loop_var: link From 7d698ec90a34c2769bceff93c9d3b64308655a4e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 30 Dec 2024 12:09:01 +0100 Subject: [PATCH 172/190] Bug fixes pr devel (#529) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Fixing linting errors around white space. * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Clearing stat and apcu cache for PHP by default. * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Moving installation of cachetool to application includes, where they are PHP apps. * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Fixing minor typo. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. --------- Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- .github/workflows/ce-deploy-test-drupal8.yml | 2 +- .github/workflows/ce-deploy-test-drupal9.yml | 2 +- ce-dev/ce-dev.compose.yml | 6 ++++++ roles/_exit/tasks/main.yml | 1 + roles/_init/tasks/main.yml | 1 + roles/deploy_code/tasks/cleanup.yml | 1 + roles/npm/tasks/main.yml | 8 ++++---- .../sync/database_sync/database_sync-mysql/tasks/sync.yml | 2 +- 8 files changed, 16 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ce-deploy-test-drupal8.yml b/.github/workflows/ce-deploy-test-drupal8.yml index ec9909a9..5ebfebfa 100644 --- a/.github/workflows/ce-deploy-test-drupal8.yml +++ b/.github/workflows/ce-deploy-test-drupal8.yml @@ -29,7 +29,7 @@ jobs: sudo mv ./mkcert /usr/local/bin && cd ../ sudo chmod +x /usr/local/bin/mkcert rm -Rf mkcert - curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/1.x/install.sh | /bin/sh -s -- linux + curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/2.x/install.sh | /bin/sh -s -- --platform linux # Uses the ce-dev stack to run a test Drupal deploy - name: Run a test ce-dev deploy with Drupal 8 diff --git a/.github/workflows/ce-deploy-test-drupal9.yml b/.github/workflows/ce-deploy-test-drupal9.yml index afaaca7a..c16bff5f 100644 --- a/.github/workflows/ce-deploy-test-drupal9.yml +++ b/.github/workflows/ce-deploy-test-drupal9.yml @@ -29,7 +29,7 @@ jobs: sudo mv ./mkcert /usr/local/bin && cd ../ sudo chmod +x /usr/local/bin/mkcert rm -Rf mkcert - curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/1.x/install.sh | /bin/sh -s -- linux + curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/2.x/install.sh | /bin/sh -s -- --platform linux # Uses the ce-dev stack to run a test Drupal deploy - name: Run a test ce-dev deploy with Drupal 9 diff --git a/ce-dev/ce-dev.compose.yml b/ce-dev/ce-dev.compose.yml index c7ff7a0e..742eb550 100644 --- a/ce-dev/ce-dev.compose.yml +++ b/ce-dev/ce-dev.compose.yml @@ -11,6 +11,8 @@ services: x-ce_dev: {} volumes: - ../:/home/ce-dev/ce-deploy + platform: linux/amd64 + cgroup: host web: image: codeenigma/ce-dev-1.x x-ce_dev: @@ -21,7 +23,11 @@ services: - 443 volumes: - ../ce-dev/build/:/home/ce-dev/deploy/live.local:delegated + platform: linux/amd64 + cgroup: host db: image: mariadb environment: MYSQL_ROOT_PASSWORD: ce-dev + platform: linux/amd64 + cgroup: host diff --git a/roles/_exit/tasks/main.yml b/roles/_exit/tasks/main.yml index 640f3a18..981986e0 100644 --- a/roles/_exit/tasks/main.yml +++ b/roles/_exit/tasks/main.yml @@ -4,6 +4,7 @@ ansible.builtin.command: > aws autoscaling resume-processes --auto-scaling-group-name {{ aws_asg.name }} --region {{ aws_asg.region }} delegate_to: localhost + run_once: true when: - aws_asg.name is defined - aws_asg.name | length > 0 diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 6f0cd830..445655c5 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -160,6 +160,7 @@ ansible.builtin.command: > aws autoscaling suspend-processes --auto-scaling-group-name {{ aws_asg.name }} --scaling-processes {{ aws_asg.suspend_processes }} --region {{ aws_asg.region }} delegate_to: localhost + run_once: true when: - aws_asg.name is defined - aws_asg.name | length > 0 diff --git a/roles/deploy_code/tasks/cleanup.yml b/roles/deploy_code/tasks/cleanup.yml index 449c1103..8d0d25b7 100644 --- a/roles/deploy_code/tasks/cleanup.yml +++ b/roles/deploy_code/tasks/cleanup.yml @@ -84,6 +84,7 @@ ansible.builtin.stat: path: "{{ build_base_path }}/deploy.sqsh" register: _deploy_code_mount_image + run_once: true - name: Copy previous SquashFS image to shared mount point in case of rollback. ansible.builtin.copy: diff --git a/roles/npm/tasks/main.yml b/roles/npm/tasks/main.yml index 09487313..eacba6c2 100644 --- a/roles/npm/tasks/main.yml +++ b/roles/npm/tasks/main.yml @@ -1,6 +1,6 @@ --- - name: Perform NPM steps. - command: + ansible.builtin.command: cmd: "{{ npm.executor }} {{ npm_command }}" chdir: "{{ npm.working_dir }}" with_items: "{{ npm.commands }}" @@ -9,7 +9,7 @@ when: deploy_operation == 'deploy' - name: Generate additional templates. - template: + ansible.builtin.template: src: "{{ template.src }}" dest: "{{ deploy_path }}/{{ template.dest }}" with_items: "{{ npm.templates }}" @@ -20,7 +20,7 @@ - deploy_operation == 'deploy' - name: Create additional symlinks. - file: + ansible.builtin.file: src: "{{ link.src }}" dest: "{{ deploy_path }}/{{ link.dest }}" state: link @@ -30,4 +30,4 @@ loop_var: link when: - npm.symlinks | length - - deploy_operation == 'deploy' \ No newline at end of file + - deploy_operation == 'deploy' diff --git a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml index 6b9d7dcd..ec36623b 100644 --- a/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml +++ b/roles/sync/database_sync/database_sync-mysql/tasks/sync.yml @@ -98,7 +98,7 @@ - not database.source.fresh_db - name: Register source database host. - set_fact: + ansible.builtin.set_fact: mysql_sync_source_database_host: "{{ mysql_host_info_grep.stdout.split('=')[1] }}" when: - not database.source.fresh_db From 4b267de09112a489d6b5dee65a390808cb701918 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 30 Dec 2024 12:28:22 +0100 Subject: [PATCH 173/190] Mautic updates pr devel (#532) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- .../deploy_code-mautic/tasks/main.yml | 23 ++++++------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/roles/deploy_code/deploy_code-mautic/tasks/main.yml b/roles/deploy_code/deploy_code-mautic/tasks/main.yml index 2ad14359..c9a60df3 100644 --- a/roles/deploy_code/deploy_code-mautic/tasks/main.yml +++ b/roles/deploy_code/deploy_code-mautic/tasks/main.yml @@ -1,8 +1,14 @@ -- name: Ensure public files directory exists. +--- +- name: Ensure public files source directory exists. file: path: "{{ deploy_assets_base_path }}/{{ project_name }}_{{ build_type }}_public_files" state: directory +- name: Ensure public files destination directory exists. + file: + path: "{{ deploy_path }}/{{ webroot }}/{{ mautic.image_path }}" + state: directory + - name: Ensure public files directory permissions. file: path: "{{ deploy_assets_base_path }}/{{ project_name }}_{{ build_type }}_public_files" @@ -13,21 +19,6 @@ become: true when: "www_user != deploy_user" -- name: Check if we have a commited public folder. - stat: - path: "{{ deploy_path }}/{{ webroot }}/{{ mautic.image_path }}" - register: _mautic_public_dir_commited - -- name: Synchronize existing assets. - command: - cmd: "rsync -av {{ deploy_path }}/{{ webroot }}/{{ mautic.image_path }}/ {{ deploy_assets_base_path }}/{{ project_name }}_{{ build_type }}_public_files" - when: _mautic_public_dir_commited.stat.isdir is defined and _mautic_public_dir_commited.stat.isdir - -- name: Remove commited public dir. - file: - path: "{{ deploy_path }}/{{ webroot }}/{{ mautic.image_path }}" - state: absent - - name: Create symlink to public files. file: src: "{{ deploy_assets_base_path }}/{{ project_name }}_{{ build_type }}_public_files" From 9c4a1d2a1eabc6f17ca70ca7aafb523ea9910dd5 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Mon, 30 Dec 2024 12:38:39 +0100 Subject: [PATCH 174/190] Mautic updates pr devel (#533) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- roles/_meta/deploy-mautic/tasks/main.yml | 2 +- roles/cache_clear/cache_clear-mautic/README.md | 7 ------- roles/cache_clear/cache_clear-mautic/tasks/main.yml | 10 ---------- .../database_apply-mautic/tasks/main.yml | 12 ++++++------ 4 files changed, 7 insertions(+), 24 deletions(-) delete mode 100644 roles/cache_clear/cache_clear-mautic/README.md delete mode 100644 roles/cache_clear/cache_clear-mautic/tasks/main.yml diff --git a/roles/_meta/deploy-mautic/tasks/main.yml b/roles/_meta/deploy-mautic/tasks/main.yml index 4ec69f31..0bf4723f 100644 --- a/roles/_meta/deploy-mautic/tasks/main.yml +++ b/roles/_meta/deploy-mautic/tasks/main.yml @@ -23,7 +23,7 @@ - ansible.builtin.import_role: name: cache_clear/cache_clear-opcache - ansible.builtin.import_role: - name: cache_clear/cache_clear-mautic + name: cache_clear/cache_clear-symfony # - ansible.builtin.import_role: # name: maintenance_mode # vars: diff --git a/roles/cache_clear/cache_clear-mautic/README.md b/roles/cache_clear/cache_clear-mautic/README.md deleted file mode 100644 index 16360291..00000000 --- a/roles/cache_clear/cache_clear-mautic/README.md +++ /dev/null @@ -1,7 +0,0 @@ -# Drupal 8 -Clear caches for Drupal 8. - - - - - diff --git a/roles/cache_clear/cache_clear-mautic/tasks/main.yml b/roles/cache_clear/cache_clear-mautic/tasks/main.yml deleted file mode 100644 index 56884d16..00000000 --- a/roles/cache_clear/cache_clear-mautic/tasks/main.yml +++ /dev/null @@ -1,10 +0,0 @@ ---- -- name: set console path - set_fact: - mautic_console_bin: "env php {{ deploy_path }}/{{ webroot }}/app/console" - -- name: Clear cache. - command: - cmd: "{{ mautic_console_bin }} cache:clear --no-interaction --env=prod --no-warmup" - become: "{{ 'no' if www_user == deploy_user else 'yes' }}" - become_user: "{{ www_user }}" diff --git a/roles/database_apply/database_apply-mautic/tasks/main.yml b/roles/database_apply/database_apply-mautic/tasks/main.yml index ec806f33..66e52472 100644 --- a/roles/database_apply/database_apply-mautic/tasks/main.yml +++ b/roles/database_apply/database_apply-mautic/tasks/main.yml @@ -1,7 +1,7 @@ --- -- name: set console path - set_fact: - mautic_console_bin: "env php {{ deploy_path }}/{{ webroot }}/app/console" +- name: Set console path. + ansible.builtin.set_fact: + symfony_console_bin: "{{ symfony_console_bin | default('env php ' + deploy_path + '/bin/console') }}" - name: Fix file permissions for local.php. file: @@ -16,19 +16,19 @@ - name: Install Mautic. command: - cmd: "{{ mautic_console_bin }} mautic:install:data --no-interaction --force" + cmd: "{{ symfony_console_bin }} mautic:install:data --no-interaction --force" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" when: previous_build_number == 0 or (mautic.force_install is defined and mautic.force_install) - name: Clear cache. command: - cmd: "{{ mautic_console_bin }} cache:clear --no-interaction --env=prod --no-warmup" + cmd: "{{ symfony_console_bin }} cache:clear --no-interaction --env=prod --no-warmup" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" - name: Apply Mautic database updates. command: - cmd: "{{ mautic_console_bin }} doctrine:migrations:migrate --no-interaction --env=prod" + cmd: "{{ symfony_console_bin }} doctrine:migrations:migrate --no-interaction --env=prod" become: "{{ 'no' if www_user == deploy_user else 'yes' }}" become_user: "{{ www_user }}" From 170ce5f57c236c33d4a78c67c8abba057617acac Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 7 Jan 2025 17:00:01 +0100 Subject: [PATCH 175/190] Fixing docs publishing. --- .github/workflows/ce-deploy-publish-docs.yml | 84 ++++++++++---------- contribute/toc.sh | 4 +- 2 files changed, 46 insertions(+), 42 deletions(-) diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index 45148600..df31e516 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -1,11 +1,12 @@ name: Publish documentation -# Run this workflow every time a new commit pushed to your repository +# Run this workflow on demand or every time a PR is accepted to a main branch on: pull_request: types: [closed] branches: - 1.x + workflow_dispatch: jobs: # Set the job key. The key is displayed as the job name @@ -14,51 +15,54 @@ jobs: # Name the Job name: Publish ce-deploy documentation # Set the type of machine to run on - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest + + # Use our ce-dev Debian base container + container: + image: codeenigma/ce-dev-controller:2.x + volumes: + - ${{ github.workspace }}:/home/controller steps: - # Checks out a copy of your repository on the ubuntu-latest machine - - name: Checkout code - uses: actions/checkout@v2 + - name: Install wiki2pages + run: /usr/bin/su - ce-dev -c "/usr/bin/git clone https://github.com/codeenigma/wikis2pages.git /home/ce-dev/build/wiki2pages" + + - name: Update local applications + run: | + /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-provision && /usr/bin/git pull origin 2.x" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-deploy && /usr/bin/git pull origin 1.x" - # Configures global Git variables for committing - - name: Configure Git + - name: Set up Ansible hosts file run: | - git config --global user.email "sysadm@codeenigma.com" - git config --global user.name "Code Enigma CI" + mkdir -p /home/ce-dev/ansible/bin/hosts + echo "wikis2pages-hugo ansible_host=127.0.0.1" > /home/ce-dev/ansible/bin/hosts/hosts - # Installs the ce-dev stack - - name: Install ce-dev + - name: Set up SSH config run: | - cd /tmp - wget https://golang.org/dl/go1.16.3.linux-amd64.tar.gz - sudo tar -C /usr/local -xzf go1.16.3.linux-amd64.tar.gz - export PATH=$PATH:/usr/local/go/bin - git clone https://github.com/FiloSottile/mkcert && cd mkcert - go build -ldflags "-X main.Version=$(git describe --tags)" - sudo mv ./mkcert /usr/local/bin && cd ../ - sudo chmod +x /usr/local/bin/mkcert - rm -Rf mkcert - curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/1.x/install.sh | /bin/sh -s -- linux + echo "StrictHostKeyChecking=no" > /home/ce-dev/.ssh/config + cat /home/ce-dev/.ssh/id_rsa.pub > /home/ce-dev/.ssh/authorized_keys + chown ce-dev:ce-dev /home/ce-dev/.ssh/config + chmod 700 /home/ce-dev/.ssh/config + chown ce-dev:ce-dev /home/ce-dev/.ssh/authorized_keys + chmod 700 /home/ce-dev/.ssh/authorized_keys + + - name: Start SSHD + run: /usr/sbin/sshd& - # Builds the table of contents for the docs - - name: Documentation (build table of contents) - if: ${{ github.event.pull_request.base.ref == '1.x' }} - run: contribute/toc.sh - shell: bash + - name: Initialise wiki2pages for ce-deploy ${{ github.event.pull_request.base.ref }} + run: | + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh init.sh --repo https://github.com/codeenigma/ce-deploy.git --branch ${{ github.event.pull_request.base.ref }} --no-ce-dev" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/provision.yml" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh set-current.sh --project ce-deploy-${{ github.event.pull_request.base.ref }} --no-ce-dev" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages launch_hugo_server=false' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/deploy.yml" + + - name: Run Hugo + run: | + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && hugo" + ls -la /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }}/ - # Uses the ce-dev stack to run Hugo to build and deploy the docs - - name: Documentation (build and publish) - if: ${{ github.event.pull_request.base.ref == '1.x' }} + - name: Publish documentation run: | - cd - git clone https://github.com/codeenigma/wikis2pages.git - cd wikis2pages - /bin/bash init.sh https://${{ secrets.GITHUB_TOKEN }}@github.com/codeenigma/ce-deploy.git 1.x - /bin/sh set-current.sh ce-deploy-1.x-travis - docker exec --user ce-dev --workdir /home/ce-dev/deploy/live.local wikis2pages-hugo hugo - /bin/sh .github-actions-push.sh - cd /home/runner/wikis2pages/public/ce-deploy-1.x-travis - git remote add ci https://${{ secrets.DOCS_GITHUB_TOKEN }}@github.com/codeenigma/ce-deploy-docs.git - git push ci master - shell: bash + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh /home/ce-dev/build/wiki2pages/.github-actions-push.sh" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }} && /usr/bin/git remote add ci https://${{ secrets.DOCS_GITHUB_TOKEN }}@github.com/codeenigma/ce-deploy-docs.git" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }} && /usr/bin/git push ci master" diff --git a/contribute/toc.sh b/contribute/toc.sh index face006c..8dc9a1c8 100755 --- a/contribute/toc.sh +++ b/contribute/toc.sh @@ -114,7 +114,7 @@ parse_roles_toc(){ "# "*) if [ "$WRITE" = "true" ]; then TITLE=$(echo "$LINE" | cut -c 3-) - echo "$INDENT"" - [$TITLE](/$RELATIVE)" >> "$TMP_SIDEBAR" + echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" WRITE="false" fi ;; @@ -137,4 +137,4 @@ generate_roles_toc cp_single_page install cp_single_page contribute -cp_single_page scripts \ No newline at end of file +cp_single_page scripts From d21c717d4d3c64c61973e98d6c0d9933353fe4c3 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 7 Jan 2025 17:01:05 +0100 Subject: [PATCH 176/190] Updating docs. --- docs/_Sidebar.md | 95 ++++++++++---------- docs/roles/cache_clear/cache_clear-mautic.md | 7 -- 2 files changed, 47 insertions(+), 55 deletions(-) delete mode 100644 docs/roles/cache_clear/cache_clear-mautic.md diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index b1f28b08..a50bee09 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -1,50 +1,49 @@   -- [Home](/) - - [Install](/install) - - [Usage](/scripts) +- [Home]() + - [Install](install) + - [Usage](scripts) - [Roles](roles) - - [API call](/roles/api_call) - - [Config](/roles/cache_clear) - - [Drupal 7](/roles/cache_clear/cache_clear-drupal7) - - [Drupal 8](/roles/cache_clear/cache_clear-drupal8) - - [Matomo](/roles/cache_clear/cache_clear-matomo) - - [Drupal 8](/roles/cache_clear/cache_clear-mautic) - - [Opcache](/roles/cache_clear/cache_clear-opcache) - - [Symfony](/roles/cache_clear/cache_clear-symfony) - - [ce-dev](/roles/ce_dev) - - [ce-dev prebuild](/roles/ce_dev/ce_dev_prebuild) - - [CLI Tools](/roles/cli) - - [Drush](/roles/cli/cachetool) - - [Drush](/roles/cli/drush) - - [Composer](/roles/composer) - - [Config](/roles/config_generate) - - [Drupal 7](/roles/config_generate/config_generate-drupal7) - - [Drupal 8](/roles/config_generate/config_generate-drupal8) - - [Drupal 8](/roles/config_generate/config_generate-matomo) - - [Mautic](/roles/config_generate/config_generate-mautic) - - [Cron](/roles/cron) - - [Database backup cron task](/roles/cron/cron_database_backup) - - [Database backup cron task - MySQL](/roles/cron/cron_database_backup/cron_database_backup-mysql) - - [Drupal 7 cron](/roles/cron/cron_drupal7) - - [Drupal 8 cron](/roles/cron/cron_drupal8) - - [Database backup cron task - MySQL](/roles/cron/cron_matomo) - - [Mautic cron tasks](/roles/cron/cron_mautic) - - [Data backups](/roles/database_backup) - - [MySQL backups](/roles/database_backup/database_backup-mysql) - - [Deploy](/roles/deploy_code) - - [Deploy container](/roles/deploy_container) - - [Exit](/roles/_exit) - - [Init](/roles/_init) - - [LHCI run](/roles/lhci_run) - - [Maintenance Mode](/roles/maintenance_mode) - - ["Meta"](/roles/_meta) - - [Drupal 10](/roles/_meta/deploy-drupal10) - - [Drupal 7](/roles/_meta/deploy-drupal7) - - [Drupal 8](/roles/_meta/deploy-drupal8) - - [Matomo](/roles/_meta/deploy-matomo) - - [Mautic](/roles/_meta/deploy-mautic) - - [SimpleSAMLphp](/roles/_meta/deploy-simplesamlphp) - - [NPM](/roles/npm) - - [Sync roles](/roles/sync) - - [Database sync](/roles/sync/database_sync) - - [Database sync - MySQL](/roles/sync/database_sync/database_sync-mysql) + - [API call](roles/api_call) + - [Config](roles/cache_clear) + - [Drupal 7](roles/cache_clear/cache_clear-drupal7) + - [Drupal 8](roles/cache_clear/cache_clear-drupal8) + - [Matomo](roles/cache_clear/cache_clear-matomo) + - [Opcache](roles/cache_clear/cache_clear-opcache) + - [Symfony](roles/cache_clear/cache_clear-symfony) + - [ce-dev](roles/ce_dev) + - [ce-dev prebuild](roles/ce_dev/ce_dev_prebuild) + - [CLI Tools](roles/cli) + - [Drush](roles/cli/cachetool) + - [Drush](roles/cli/drush) + - [Composer](roles/composer) + - [Config](roles/config_generate) + - [Drupal 7](roles/config_generate/config_generate-drupal7) + - [Drupal 8](roles/config_generate/config_generate-drupal8) + - [Drupal 8](roles/config_generate/config_generate-matomo) + - [Mautic](roles/config_generate/config_generate-mautic) + - [Cron](roles/cron) + - [Database backup cron task](roles/cron/cron_database_backup) + - [Database backup cron task - MySQL](roles/cron/cron_database_backup/cron_database_backup-mysql) + - [Drupal 7 cron](roles/cron/cron_drupal7) + - [Drupal 8 cron](roles/cron/cron_drupal8) + - [Database backup cron task - MySQL](roles/cron/cron_matomo) + - [Mautic cron tasks](roles/cron/cron_mautic) + - [Data backups](roles/database_backup) + - [MySQL backups](roles/database_backup/database_backup-mysql) + - [Deploy](roles/deploy_code) + - [Deploy container](roles/deploy_container) + - [Exit](roles/_exit) + - [Init](roles/_init) + - [LHCI run](roles/lhci_run) + - [Maintenance Mode](roles/maintenance_mode) + - ["Meta"](roles/_meta) + - [Drupal 10](roles/_meta/deploy-drupal10) + - [Drupal 7](roles/_meta/deploy-drupal7) + - [Drupal 8](roles/_meta/deploy-drupal8) + - [Matomo](roles/_meta/deploy-matomo) + - [Mautic](roles/_meta/deploy-mautic) + - [SimpleSAMLphp](roles/_meta/deploy-simplesamlphp) + - [NPM](roles/npm) + - [Sync roles](roles/sync) + - [Database sync](roles/sync/database_sync) + - [Database sync - MySQL](roles/sync/database_sync/database_sync-mysql) diff --git a/docs/roles/cache_clear/cache_clear-mautic.md b/docs/roles/cache_clear/cache_clear-mautic.md deleted file mode 100644 index 16360291..00000000 --- a/docs/roles/cache_clear/cache_clear-mautic.md +++ /dev/null @@ -1,7 +0,0 @@ -# Drupal 8 -Clear caches for Drupal 8. - - - - - From 6fcc2eb160a1f5fa0eb2aea53917ce1b7bbc5646 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 7 Jan 2025 17:05:02 +0100 Subject: [PATCH 177/190] Publish docs pr devel (#535) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Mautic updates pr 1.x (#531) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Mautic updates pr 1.x (#534) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code. * Fixing docs publishing. * Updating docs. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- .github/workflows/ce-deploy-publish-docs.yml | 84 ++++++++--------- contribute/toc.sh | 4 +- docs/_Sidebar.md | 95 ++++++++++---------- docs/roles/cache_clear/cache_clear-mautic.md | 7 -- 4 files changed, 93 insertions(+), 97 deletions(-) delete mode 100644 docs/roles/cache_clear/cache_clear-mautic.md diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index 45148600..df31e516 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -1,11 +1,12 @@ name: Publish documentation -# Run this workflow every time a new commit pushed to your repository +# Run this workflow on demand or every time a PR is accepted to a main branch on: pull_request: types: [closed] branches: - 1.x + workflow_dispatch: jobs: # Set the job key. The key is displayed as the job name @@ -14,51 +15,54 @@ jobs: # Name the Job name: Publish ce-deploy documentation # Set the type of machine to run on - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest + + # Use our ce-dev Debian base container + container: + image: codeenigma/ce-dev-controller:2.x + volumes: + - ${{ github.workspace }}:/home/controller steps: - # Checks out a copy of your repository on the ubuntu-latest machine - - name: Checkout code - uses: actions/checkout@v2 + - name: Install wiki2pages + run: /usr/bin/su - ce-dev -c "/usr/bin/git clone https://github.com/codeenigma/wikis2pages.git /home/ce-dev/build/wiki2pages" + + - name: Update local applications + run: | + /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-provision && /usr/bin/git pull origin 2.x" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-deploy && /usr/bin/git pull origin 1.x" - # Configures global Git variables for committing - - name: Configure Git + - name: Set up Ansible hosts file run: | - git config --global user.email "sysadm@codeenigma.com" - git config --global user.name "Code Enigma CI" + mkdir -p /home/ce-dev/ansible/bin/hosts + echo "wikis2pages-hugo ansible_host=127.0.0.1" > /home/ce-dev/ansible/bin/hosts/hosts - # Installs the ce-dev stack - - name: Install ce-dev + - name: Set up SSH config run: | - cd /tmp - wget https://golang.org/dl/go1.16.3.linux-amd64.tar.gz - sudo tar -C /usr/local -xzf go1.16.3.linux-amd64.tar.gz - export PATH=$PATH:/usr/local/go/bin - git clone https://github.com/FiloSottile/mkcert && cd mkcert - go build -ldflags "-X main.Version=$(git describe --tags)" - sudo mv ./mkcert /usr/local/bin && cd ../ - sudo chmod +x /usr/local/bin/mkcert - rm -Rf mkcert - curl -sL https://raw.githubusercontent.com/codeenigma/ce-dev/1.x/install.sh | /bin/sh -s -- linux + echo "StrictHostKeyChecking=no" > /home/ce-dev/.ssh/config + cat /home/ce-dev/.ssh/id_rsa.pub > /home/ce-dev/.ssh/authorized_keys + chown ce-dev:ce-dev /home/ce-dev/.ssh/config + chmod 700 /home/ce-dev/.ssh/config + chown ce-dev:ce-dev /home/ce-dev/.ssh/authorized_keys + chmod 700 /home/ce-dev/.ssh/authorized_keys + + - name: Start SSHD + run: /usr/sbin/sshd& - # Builds the table of contents for the docs - - name: Documentation (build table of contents) - if: ${{ github.event.pull_request.base.ref == '1.x' }} - run: contribute/toc.sh - shell: bash + - name: Initialise wiki2pages for ce-deploy ${{ github.event.pull_request.base.ref }} + run: | + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh init.sh --repo https://github.com/codeenigma/ce-deploy.git --branch ${{ github.event.pull_request.base.ref }} --no-ce-dev" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/provision.yml" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh set-current.sh --project ce-deploy-${{ github.event.pull_request.base.ref }} --no-ce-dev" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages launch_hugo_server=false' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/deploy.yml" + + - name: Run Hugo + run: | + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && hugo" + ls -la /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }}/ - # Uses the ce-dev stack to run Hugo to build and deploy the docs - - name: Documentation (build and publish) - if: ${{ github.event.pull_request.base.ref == '1.x' }} + - name: Publish documentation run: | - cd - git clone https://github.com/codeenigma/wikis2pages.git - cd wikis2pages - /bin/bash init.sh https://${{ secrets.GITHUB_TOKEN }}@github.com/codeenigma/ce-deploy.git 1.x - /bin/sh set-current.sh ce-deploy-1.x-travis - docker exec --user ce-dev --workdir /home/ce-dev/deploy/live.local wikis2pages-hugo hugo - /bin/sh .github-actions-push.sh - cd /home/runner/wikis2pages/public/ce-deploy-1.x-travis - git remote add ci https://${{ secrets.DOCS_GITHUB_TOKEN }}@github.com/codeenigma/ce-deploy-docs.git - git push ci master - shell: bash + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh /home/ce-dev/build/wiki2pages/.github-actions-push.sh" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }} && /usr/bin/git remote add ci https://${{ secrets.DOCS_GITHUB_TOKEN }}@github.com/codeenigma/ce-deploy-docs.git" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }} && /usr/bin/git push ci master" diff --git a/contribute/toc.sh b/contribute/toc.sh index face006c..8dc9a1c8 100755 --- a/contribute/toc.sh +++ b/contribute/toc.sh @@ -114,7 +114,7 @@ parse_roles_toc(){ "# "*) if [ "$WRITE" = "true" ]; then TITLE=$(echo "$LINE" | cut -c 3-) - echo "$INDENT"" - [$TITLE](/$RELATIVE)" >> "$TMP_SIDEBAR" + echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" WRITE="false" fi ;; @@ -137,4 +137,4 @@ generate_roles_toc cp_single_page install cp_single_page contribute -cp_single_page scripts \ No newline at end of file +cp_single_page scripts diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index b1f28b08..a50bee09 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -1,50 +1,49 @@   -- [Home](/) - - [Install](/install) - - [Usage](/scripts) +- [Home]() + - [Install](install) + - [Usage](scripts) - [Roles](roles) - - [API call](/roles/api_call) - - [Config](/roles/cache_clear) - - [Drupal 7](/roles/cache_clear/cache_clear-drupal7) - - [Drupal 8](/roles/cache_clear/cache_clear-drupal8) - - [Matomo](/roles/cache_clear/cache_clear-matomo) - - [Drupal 8](/roles/cache_clear/cache_clear-mautic) - - [Opcache](/roles/cache_clear/cache_clear-opcache) - - [Symfony](/roles/cache_clear/cache_clear-symfony) - - [ce-dev](/roles/ce_dev) - - [ce-dev prebuild](/roles/ce_dev/ce_dev_prebuild) - - [CLI Tools](/roles/cli) - - [Drush](/roles/cli/cachetool) - - [Drush](/roles/cli/drush) - - [Composer](/roles/composer) - - [Config](/roles/config_generate) - - [Drupal 7](/roles/config_generate/config_generate-drupal7) - - [Drupal 8](/roles/config_generate/config_generate-drupal8) - - [Drupal 8](/roles/config_generate/config_generate-matomo) - - [Mautic](/roles/config_generate/config_generate-mautic) - - [Cron](/roles/cron) - - [Database backup cron task](/roles/cron/cron_database_backup) - - [Database backup cron task - MySQL](/roles/cron/cron_database_backup/cron_database_backup-mysql) - - [Drupal 7 cron](/roles/cron/cron_drupal7) - - [Drupal 8 cron](/roles/cron/cron_drupal8) - - [Database backup cron task - MySQL](/roles/cron/cron_matomo) - - [Mautic cron tasks](/roles/cron/cron_mautic) - - [Data backups](/roles/database_backup) - - [MySQL backups](/roles/database_backup/database_backup-mysql) - - [Deploy](/roles/deploy_code) - - [Deploy container](/roles/deploy_container) - - [Exit](/roles/_exit) - - [Init](/roles/_init) - - [LHCI run](/roles/lhci_run) - - [Maintenance Mode](/roles/maintenance_mode) - - ["Meta"](/roles/_meta) - - [Drupal 10](/roles/_meta/deploy-drupal10) - - [Drupal 7](/roles/_meta/deploy-drupal7) - - [Drupal 8](/roles/_meta/deploy-drupal8) - - [Matomo](/roles/_meta/deploy-matomo) - - [Mautic](/roles/_meta/deploy-mautic) - - [SimpleSAMLphp](/roles/_meta/deploy-simplesamlphp) - - [NPM](/roles/npm) - - [Sync roles](/roles/sync) - - [Database sync](/roles/sync/database_sync) - - [Database sync - MySQL](/roles/sync/database_sync/database_sync-mysql) + - [API call](roles/api_call) + - [Config](roles/cache_clear) + - [Drupal 7](roles/cache_clear/cache_clear-drupal7) + - [Drupal 8](roles/cache_clear/cache_clear-drupal8) + - [Matomo](roles/cache_clear/cache_clear-matomo) + - [Opcache](roles/cache_clear/cache_clear-opcache) + - [Symfony](roles/cache_clear/cache_clear-symfony) + - [ce-dev](roles/ce_dev) + - [ce-dev prebuild](roles/ce_dev/ce_dev_prebuild) + - [CLI Tools](roles/cli) + - [Drush](roles/cli/cachetool) + - [Drush](roles/cli/drush) + - [Composer](roles/composer) + - [Config](roles/config_generate) + - [Drupal 7](roles/config_generate/config_generate-drupal7) + - [Drupal 8](roles/config_generate/config_generate-drupal8) + - [Drupal 8](roles/config_generate/config_generate-matomo) + - [Mautic](roles/config_generate/config_generate-mautic) + - [Cron](roles/cron) + - [Database backup cron task](roles/cron/cron_database_backup) + - [Database backup cron task - MySQL](roles/cron/cron_database_backup/cron_database_backup-mysql) + - [Drupal 7 cron](roles/cron/cron_drupal7) + - [Drupal 8 cron](roles/cron/cron_drupal8) + - [Database backup cron task - MySQL](roles/cron/cron_matomo) + - [Mautic cron tasks](roles/cron/cron_mautic) + - [Data backups](roles/database_backup) + - [MySQL backups](roles/database_backup/database_backup-mysql) + - [Deploy](roles/deploy_code) + - [Deploy container](roles/deploy_container) + - [Exit](roles/_exit) + - [Init](roles/_init) + - [LHCI run](roles/lhci_run) + - [Maintenance Mode](roles/maintenance_mode) + - ["Meta"](roles/_meta) + - [Drupal 10](roles/_meta/deploy-drupal10) + - [Drupal 7](roles/_meta/deploy-drupal7) + - [Drupal 8](roles/_meta/deploy-drupal8) + - [Matomo](roles/_meta/deploy-matomo) + - [Mautic](roles/_meta/deploy-mautic) + - [SimpleSAMLphp](roles/_meta/deploy-simplesamlphp) + - [NPM](roles/npm) + - [Sync roles](roles/sync) + - [Database sync](roles/sync/database_sync) + - [Database sync - MySQL](roles/sync/database_sync/database_sync-mysql) diff --git a/docs/roles/cache_clear/cache_clear-mautic.md b/docs/roles/cache_clear/cache_clear-mautic.md deleted file mode 100644 index 16360291..00000000 --- a/docs/roles/cache_clear/cache_clear-mautic.md +++ /dev/null @@ -1,7 +0,0 @@ -# Drupal 8 -Clear caches for Drupal 8. - - - - - From f778ec27e767df4c82d1f06600c719d6735c20ca Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 7 Jan 2025 17:11:52 +0100 Subject: [PATCH 178/190] Updating name of job. --- .wikis2pages.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.wikis2pages.yml b/.wikis2pages.yml index 5addb318..a282ce2d 100644 --- a/.wikis2pages.yml +++ b/.wikis2pages.yml @@ -1,4 +1,4 @@ -ce-deploy-1.x-travis: +ce-deploy-1.x: src: https://github.com/codeenigma/ce-deploy.git src_branch: 1.x src_subdir: 'docs' @@ -7,4 +7,4 @@ ce-deploy-1.x-travis: dest_subdir: 1.x title: ce-deploy type: doc - base_url: https://codeenigma.github.io/ce-deploy-docs/1.x \ No newline at end of file + base_url: https://codeenigma.github.io/ce-deploy-docs/1.x From b6aaf0e652240a9638060104979d93faa8d22047 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Tue, 7 Jan 2025 17:16:12 +0100 Subject: [PATCH 179/190] Publish docs pr devel (#537) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Mautic updates pr 1.x (#531) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Mautic updates pr 1.x (#534) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code. * Fixing docs publishing. * Updating docs. * Updating name of job. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- .wikis2pages.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.wikis2pages.yml b/.wikis2pages.yml index 5addb318..a282ce2d 100644 --- a/.wikis2pages.yml +++ b/.wikis2pages.yml @@ -1,4 +1,4 @@ -ce-deploy-1.x-travis: +ce-deploy-1.x: src: https://github.com/codeenigma/ce-deploy.git src_branch: 1.x src_subdir: 'docs' @@ -7,4 +7,4 @@ ce-deploy-1.x-travis: dest_subdir: 1.x title: ce-deploy type: doc - base_url: https://codeenigma.github.io/ce-deploy-docs/1.x \ No newline at end of file + base_url: https://codeenigma.github.io/ce-deploy-docs/1.x From 37e663d42b3884c5ecea2f125819c5f016e3379f Mon Sep 17 00:00:00 2001 From: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Date: Wed, 8 Jan 2025 11:27:08 +0200 Subject: [PATCH 180/190] Lockfile behavior bugfix pr devel (#539) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Mautic updates pr 1.x (#531) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Mautic updates pr 1.x (#534) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code. * Publish docs pr 1.x (#536) * Fixing docs publishing. * Updating docs. * Publish docs pr 1.x (#538) * Fixing docs publishing. * Updating docs. * Updating name of job. * lockfile_behavior_bugfix --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- roles/_init/tasks/main.yml | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/roles/_init/tasks/main.yml b/roles/_init/tasks/main.yml index 445655c5..34340018 100644 --- a/roles/_init/tasks/main.yml +++ b/roles/_init/tasks/main.yml @@ -8,21 +8,15 @@ path: "{{ provision_lock_file }}" register: _ce_provision_lock -- name: Abort if ce-provision lock file exists. +- name: Fail the pipeline if ce-provision lock file exists. + ansible.builtin.fail: + msg: "ce-provision lock file discovered, an infrastructure build is in progress! If this is not the case, login to the affected server and delete the file at {{ provision_lock_file }}." when: _ce_provision_lock.stat.exists is defined and _ce_provision_lock.stat.exists - block: - - name: Abort if ce-provision lock file is found. - ansible.builtin.debug: - msg: "ce-provision lock file discovered, an infrastructure build is in progress! If this is not the case, login to the affected server and delete the file at {{ provision_lock_file }}." - - ansible.builtin.meta: end_play -- name: Check OS family. +- name: Fail the pipeline if target is a Windows server. + ansible.builtin.fail: + msg: "ce-deploy currently only supports Linux like operating systems, and works best with Debian Linux." when: ansible_os_family == "Windows" - block: - - name: Abort if target is a Windows server. - ansible.builtin.debug: - msg: "ce-deploy currently only supports Linux like operating systems, and works best with Debian Linux." - - ansible.builtin.meta: end_play - name: Set a lock file. ansible.builtin.file: From 65bfbca52d3bcaa3bd9d41ea87127b74f0c5236e Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 8 Jan 2025 13:02:59 +0100 Subject: [PATCH 181/190] Making docs publishing changes from ce-provision in ce-deploy. --- .github/workflows/ce-deploy-publish-docs.yml | 43 ++++-- contribute/toc.sh | 2 +- contribute/toc_hugo.sh | 140 +++++++++++++++++++ 3 files changed, 173 insertions(+), 12 deletions(-) create mode 100755 contribute/toc_hugo.sh diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index df31e516..063ef344 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -12,6 +12,8 @@ jobs: # Set the job key. The key is displayed as the job name # when a job name is not provided publish-docs: + # Only run the job if it is not coming from a documentation branch + if: ${{ github.event.pull_request.head.ref != 'docs-${{ github.event.pull_request.base.ref }}' }} # Name the Job name: Publish ce-deploy documentation # Set the type of machine to run on @@ -24,30 +26,49 @@ jobs: - ${{ github.workspace }}:/home/controller steps: - - name: Install wiki2pages - run: /usr/bin/su - ce-dev -c "/usr/bin/git clone https://github.com/codeenigma/wikis2pages.git /home/ce-dev/build/wiki2pages" + - uses: actions/checkout@v4 + with: + ref: docs-${{ github.event.pull_request.base.ref }} - - name: Update local applications + # Configure environment + - name: Prepare Git, GitHub CLI and installed CE tools run: | + /usr/bin/git config --global user.email "sysadm@codeenigma.com" + /usr/bin/git config --global user.name "Code Enigma CI" + /usr/bin/git config --global pull.rebase false + /usr/bin/git config --global --add safe.directory /__w/ce-deploy/ce-deploy + (type -p wget >/dev/null || (sudo apt update && sudo apt-get install wget -y)) && sudo mkdir -p -m 755 /etc/apt/keyrings && out=$(mktemp) && wget -nv -O$out https://cli.github.com/packages/githubcli-archive-keyring.gpg && cat $out | sudo tee /etc/apt/keyrings/githubcli-archive-keyring.gpg > /dev/null && sudo chmod go+r /etc/apt/keyrings/githubcli-archive-keyring.gpg && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null && sudo apt update && sudo apt install gh -y /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-provision && /usr/bin/git pull origin 2.x" /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-deploy && /usr/bin/git pull origin 1.x" - - name: Set up Ansible hosts file + # First build and publish the markdown docs + - name: Build and commit table of contents and README files back to the repo run: | - mkdir -p /home/ce-dev/ansible/bin/hosts - echo "wikis2pages-hugo ansible_host=127.0.0.1" > /home/ce-dev/ansible/bin/hosts/hosts + /bin/sh contribute/toc.sh + /usr/bin/find . -name "*.md" | xargs git add + /usr/bin/git diff --quiet && git diff --staged --quiet || git commit -am "GitHub Actions - updating markdown docs - ${{ steps.date.outputs.date }}" + /usr/bin/git push + + # Create docs pull request + - name: Create a documentation pull request + run: gh pr create --base ${{ github.event.pull_request.base.ref }} --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update" --body "**Automated pull request** created by GitHub Actions because of a documentation update." + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - name: Set up SSH config + - name: Install wiki2pages + run: /usr/bin/su - ce-dev -c "/usr/bin/git clone https://github.com/codeenigma/wikis2pages.git /home/ce-dev/build/wiki2pages" + + - name: Set up Ansible and SSH run: | + mkdir -p /home/ce-dev/ansible/bin/hosts + echo "wikis2pages-hugo ansible_host=127.0.0.1" > /home/ce-dev/ansible/bin/hosts/hosts echo "StrictHostKeyChecking=no" > /home/ce-dev/.ssh/config cat /home/ce-dev/.ssh/id_rsa.pub > /home/ce-dev/.ssh/authorized_keys chown ce-dev:ce-dev /home/ce-dev/.ssh/config chmod 700 /home/ce-dev/.ssh/config chown ce-dev:ce-dev /home/ce-dev/.ssh/authorized_keys chmod 700 /home/ce-dev/.ssh/authorized_keys - - - name: Start SSHD - run: /usr/sbin/sshd& + /usr/sbin/sshd& - name: Initialise wiki2pages for ce-deploy ${{ github.event.pull_request.base.ref }} run: | @@ -55,11 +76,11 @@ jobs: /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/provision.yml" /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh set-current.sh --project ce-deploy-${{ github.event.pull_request.base.ref }} --no-ce-dev" /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages launch_hugo_server=false' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/deploy.yml" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages/content/ce-deploy-${{ github.event.pull_request.base.ref }} && /bin/sh contribute/toc_hugo.sh" - name: Run Hugo run: | /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && hugo" - ls -la /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }}/ - name: Publish documentation run: | diff --git a/contribute/toc.sh b/contribute/toc.sh index 8dc9a1c8..8c108c60 100755 --- a/contribute/toc.sh +++ b/contribute/toc.sh @@ -114,7 +114,7 @@ parse_roles_toc(){ "# "*) if [ "$WRITE" = "true" ]; then TITLE=$(echo "$LINE" | cut -c 3-) - echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" + echo "$INDENT"" - [$TITLE](/$RELATIVE)" >> "$TMP_SIDEBAR" WRITE="false" fi ;; diff --git a/contribute/toc_hugo.sh b/contribute/toc_hugo.sh new file mode 100755 index 00000000..8dc9a1c8 --- /dev/null +++ b/contribute/toc_hugo.sh @@ -0,0 +1,140 @@ +#!/bin/sh +# shellcheck disable=SC2094 +# shellcheck disable=SC2129 +IFS=$(printf '\n\t') +set -e +OWN_DIR=$(dirname "$0") +cd "$OWN_DIR" || exit 1 +OWN_DIR=$(git rev-parse --show-toplevel) +cd "$OWN_DIR" || exit 1 +OWN_DIR=$(pwd -P) + +# @param +# $1 string filepath +cp_role_page(){ + RELATIVE=$(realpath --relative-to="$OWN_DIR" "$(dirname "$1")") + if [ ! -d "$OWN_DIR/docs/$RELATIVE" ]; then + mkdir -p "$OWN_DIR/docs/$RELATIVE" + fi + cp "$1" "$OWN_DIR/docs/$RELATIVE.md" +} + +# @param +# $1 string folder +cp_single_page(){ + if [ ! -d "$OWN_DIR/docs/$1" ]; then + mkdir "$OWN_DIR/docs/$1" + fi + cp "$OWN_DIR/$1/README.md" "$OWN_DIR/docs/$1.md" +} + +# @param +# $1 (string) filename +parse_role_variables(){ + TMP_MD=$(mktemp) + WRITE=1 + # Ensure we have a trailing line. + echo "" >> "$1" + while read -r LINE; do + case $LINE in + '') + echo "$LINE" >> "$TMP_MD" + generate_role_variables "$1" + WRITE=0 + ;; + '') + echo "$LINE" >> "$TMP_MD" + WRITE=1 + ;; + '') + echo "$LINE" >> "$TMP_MD" + WRITE=0 + ;; + '') + echo "$LINE" >> "$TMP_MD" + WRITE=1 + ;; + *) + if [ $WRITE = 1 ]; then + echo "$LINE" >> "$TMP_MD" + fi + ;; + esac + done < "$1" + printf '%s\n' "$(cat "$TMP_MD")" > "$1" + rm "$TMP_MD" +} + +# @param +# $1 (string) filename +generate_role_variables(){ + VAR_FILE="$(dirname "$1")/defaults/main.yml" + if [ -f "$VAR_FILE" ]; then + echo "## Default variables" >> "$TMP_MD" + echo '```yaml' >> "$TMP_MD" + cat "$VAR_FILE" >> "$TMP_MD" + echo "" >> "$TMP_MD" + echo '```' >> "$TMP_MD" + echo "" >> "$TMP_MD" + fi +} + +generate_roles_toc(){ + TMP_SIDEBAR=$(mktemp) + WRITE="true" + while read -r LINE; do + case $LINE in + " - [Roles](roles)") + echo "$LINE" >> "$TMP_SIDEBAR" + parse_roles_toc roles 2 + WRITE="false" + ;; + " -"*) + WRITE="true" + echo "$LINE" >> "$TMP_SIDEBAR" + ;; + *) + if [ "$WRITE" = "true" ]; then + echo "$LINE" >> "$TMP_SIDEBAR" + fi + ;; + esac + done < "$OWN_DIR/docs/_Sidebar.md" + mv "$TMP_SIDEBAR" "$OWN_DIR/docs/_Sidebar.md" +} + +parse_roles_toc(){ + ROLES=$(find "$OWN_DIR/$1" -mindepth 2 -maxdepth 2 -name "README.md" | sort) + for ROLE in $ROLES; do + WRITE="true" + INDENT=$(printf %$(($2 * 2))s) + RELATIVE=$(realpath --relative-to="$OWN_DIR" "$(dirname "$ROLE")") + while read -r LINE; do + case $LINE in + "# "*) + if [ "$WRITE" = "true" ]; then + TITLE=$(echo "$LINE" | cut -c 3-) + echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" + WRITE="false" + fi + ;; + esac + done < "$ROLE" + parse_roles_toc "$RELATIVE" $(($2 + 1)) + done +} + +rm -rf "$OWN_DIR/docs/roles" +ROLE_PAGES=$(find "$OWN_DIR/roles" -name "README.md") +for ROLE_PAGE in $ROLE_PAGES; do + parse_role_variables "$ROLE_PAGE" +done +for ROLE_PAGE in $ROLE_PAGES; do + cp_role_page "$ROLE_PAGE" +done +generate_roles_toc + + +cp_single_page install +cp_single_page contribute +cp_single_page scripts From 120f9e73f145dfe31deb894e9aaf6ab549d13603 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 8 Jan 2025 13:04:35 +0100 Subject: [PATCH 182/190] Publish docs pr devel (#541) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Mautic updates pr 1.x (#531) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Mautic updates pr 1.x (#534) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code. * Fixing docs publishing. * Updating docs. * Publish docs pr 1.x (#536) * Fixing docs publishing. * Updating docs. * Updating name of job. * Publish docs pr 1.x (#538) * Fixing docs publishing. * Updating docs. * Updating name of job. * lockfile_behavior_bugfix (#540) * Making docs publishing changes from ce-provision in ce-deploy. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- .github/workflows/ce-deploy-publish-docs.yml | 43 ++++-- contribute/toc.sh | 2 +- contribute/toc_hugo.sh | 140 +++++++++++++++++++ 3 files changed, 173 insertions(+), 12 deletions(-) create mode 100755 contribute/toc_hugo.sh diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index df31e516..063ef344 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -12,6 +12,8 @@ jobs: # Set the job key. The key is displayed as the job name # when a job name is not provided publish-docs: + # Only run the job if it is not coming from a documentation branch + if: ${{ github.event.pull_request.head.ref != 'docs-${{ github.event.pull_request.base.ref }}' }} # Name the Job name: Publish ce-deploy documentation # Set the type of machine to run on @@ -24,30 +26,49 @@ jobs: - ${{ github.workspace }}:/home/controller steps: - - name: Install wiki2pages - run: /usr/bin/su - ce-dev -c "/usr/bin/git clone https://github.com/codeenigma/wikis2pages.git /home/ce-dev/build/wiki2pages" + - uses: actions/checkout@v4 + with: + ref: docs-${{ github.event.pull_request.base.ref }} - - name: Update local applications + # Configure environment + - name: Prepare Git, GitHub CLI and installed CE tools run: | + /usr/bin/git config --global user.email "sysadm@codeenigma.com" + /usr/bin/git config --global user.name "Code Enigma CI" + /usr/bin/git config --global pull.rebase false + /usr/bin/git config --global --add safe.directory /__w/ce-deploy/ce-deploy + (type -p wget >/dev/null || (sudo apt update && sudo apt-get install wget -y)) && sudo mkdir -p -m 755 /etc/apt/keyrings && out=$(mktemp) && wget -nv -O$out https://cli.github.com/packages/githubcli-archive-keyring.gpg && cat $out | sudo tee /etc/apt/keyrings/githubcli-archive-keyring.gpg > /dev/null && sudo chmod go+r /etc/apt/keyrings/githubcli-archive-keyring.gpg && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null && sudo apt update && sudo apt install gh -y /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-provision && /usr/bin/git pull origin 2.x" /usr/bin/su - ce-dev -c "cd /home/ce-dev/ce-deploy && /usr/bin/git pull origin 1.x" - - name: Set up Ansible hosts file + # First build and publish the markdown docs + - name: Build and commit table of contents and README files back to the repo run: | - mkdir -p /home/ce-dev/ansible/bin/hosts - echo "wikis2pages-hugo ansible_host=127.0.0.1" > /home/ce-dev/ansible/bin/hosts/hosts + /bin/sh contribute/toc.sh + /usr/bin/find . -name "*.md" | xargs git add + /usr/bin/git diff --quiet && git diff --staged --quiet || git commit -am "GitHub Actions - updating markdown docs - ${{ steps.date.outputs.date }}" + /usr/bin/git push + + # Create docs pull request + - name: Create a documentation pull request + run: gh pr create --base ${{ github.event.pull_request.base.ref }} --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update" --body "**Automated pull request** created by GitHub Actions because of a documentation update." + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - name: Set up SSH config + - name: Install wiki2pages + run: /usr/bin/su - ce-dev -c "/usr/bin/git clone https://github.com/codeenigma/wikis2pages.git /home/ce-dev/build/wiki2pages" + + - name: Set up Ansible and SSH run: | + mkdir -p /home/ce-dev/ansible/bin/hosts + echo "wikis2pages-hugo ansible_host=127.0.0.1" > /home/ce-dev/ansible/bin/hosts/hosts echo "StrictHostKeyChecking=no" > /home/ce-dev/.ssh/config cat /home/ce-dev/.ssh/id_rsa.pub > /home/ce-dev/.ssh/authorized_keys chown ce-dev:ce-dev /home/ce-dev/.ssh/config chmod 700 /home/ce-dev/.ssh/config chown ce-dev:ce-dev /home/ce-dev/.ssh/authorized_keys chmod 700 /home/ce-dev/.ssh/authorized_keys - - - name: Start SSHD - run: /usr/sbin/sshd& + /usr/sbin/sshd& - name: Initialise wiki2pages for ce-deploy ${{ github.event.pull_request.base.ref }} run: | @@ -55,11 +76,11 @@ jobs: /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/provision.yml" /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh set-current.sh --project ce-deploy-${{ github.event.pull_request.base.ref }} --no-ce-dev" /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages launch_hugo_server=false' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/deploy.yml" + /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages/content/ce-deploy-${{ github.event.pull_request.base.ref }} && /bin/sh contribute/toc_hugo.sh" - name: Run Hugo run: | /usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && hugo" - ls -la /home/ce-dev/build/wiki2pages/public/ce-deploy-${{ github.event.pull_request.base.ref }}/ - name: Publish documentation run: | diff --git a/contribute/toc.sh b/contribute/toc.sh index 8dc9a1c8..8c108c60 100755 --- a/contribute/toc.sh +++ b/contribute/toc.sh @@ -114,7 +114,7 @@ parse_roles_toc(){ "# "*) if [ "$WRITE" = "true" ]; then TITLE=$(echo "$LINE" | cut -c 3-) - echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" + echo "$INDENT"" - [$TITLE](/$RELATIVE)" >> "$TMP_SIDEBAR" WRITE="false" fi ;; diff --git a/contribute/toc_hugo.sh b/contribute/toc_hugo.sh new file mode 100755 index 00000000..8dc9a1c8 --- /dev/null +++ b/contribute/toc_hugo.sh @@ -0,0 +1,140 @@ +#!/bin/sh +# shellcheck disable=SC2094 +# shellcheck disable=SC2129 +IFS=$(printf '\n\t') +set -e +OWN_DIR=$(dirname "$0") +cd "$OWN_DIR" || exit 1 +OWN_DIR=$(git rev-parse --show-toplevel) +cd "$OWN_DIR" || exit 1 +OWN_DIR=$(pwd -P) + +# @param +# $1 string filepath +cp_role_page(){ + RELATIVE=$(realpath --relative-to="$OWN_DIR" "$(dirname "$1")") + if [ ! -d "$OWN_DIR/docs/$RELATIVE" ]; then + mkdir -p "$OWN_DIR/docs/$RELATIVE" + fi + cp "$1" "$OWN_DIR/docs/$RELATIVE.md" +} + +# @param +# $1 string folder +cp_single_page(){ + if [ ! -d "$OWN_DIR/docs/$1" ]; then + mkdir "$OWN_DIR/docs/$1" + fi + cp "$OWN_DIR/$1/README.md" "$OWN_DIR/docs/$1.md" +} + +# @param +# $1 (string) filename +parse_role_variables(){ + TMP_MD=$(mktemp) + WRITE=1 + # Ensure we have a trailing line. + echo "" >> "$1" + while read -r LINE; do + case $LINE in + '') + echo "$LINE" >> "$TMP_MD" + generate_role_variables "$1" + WRITE=0 + ;; + '') + echo "$LINE" >> "$TMP_MD" + WRITE=1 + ;; + '') + echo "$LINE" >> "$TMP_MD" + WRITE=0 + ;; + '') + echo "$LINE" >> "$TMP_MD" + WRITE=1 + ;; + *) + if [ $WRITE = 1 ]; then + echo "$LINE" >> "$TMP_MD" + fi + ;; + esac + done < "$1" + printf '%s\n' "$(cat "$TMP_MD")" > "$1" + rm "$TMP_MD" +} + +# @param +# $1 (string) filename +generate_role_variables(){ + VAR_FILE="$(dirname "$1")/defaults/main.yml" + if [ -f "$VAR_FILE" ]; then + echo "## Default variables" >> "$TMP_MD" + echo '```yaml' >> "$TMP_MD" + cat "$VAR_FILE" >> "$TMP_MD" + echo "" >> "$TMP_MD" + echo '```' >> "$TMP_MD" + echo "" >> "$TMP_MD" + fi +} + +generate_roles_toc(){ + TMP_SIDEBAR=$(mktemp) + WRITE="true" + while read -r LINE; do + case $LINE in + " - [Roles](roles)") + echo "$LINE" >> "$TMP_SIDEBAR" + parse_roles_toc roles 2 + WRITE="false" + ;; + " -"*) + WRITE="true" + echo "$LINE" >> "$TMP_SIDEBAR" + ;; + *) + if [ "$WRITE" = "true" ]; then + echo "$LINE" >> "$TMP_SIDEBAR" + fi + ;; + esac + done < "$OWN_DIR/docs/_Sidebar.md" + mv "$TMP_SIDEBAR" "$OWN_DIR/docs/_Sidebar.md" +} + +parse_roles_toc(){ + ROLES=$(find "$OWN_DIR/$1" -mindepth 2 -maxdepth 2 -name "README.md" | sort) + for ROLE in $ROLES; do + WRITE="true" + INDENT=$(printf %$(($2 * 2))s) + RELATIVE=$(realpath --relative-to="$OWN_DIR" "$(dirname "$ROLE")") + while read -r LINE; do + case $LINE in + "# "*) + if [ "$WRITE" = "true" ]; then + TITLE=$(echo "$LINE" | cut -c 3-) + echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" + WRITE="false" + fi + ;; + esac + done < "$ROLE" + parse_roles_toc "$RELATIVE" $(($2 + 1)) + done +} + +rm -rf "$OWN_DIR/docs/roles" +ROLE_PAGES=$(find "$OWN_DIR/roles" -name "README.md") +for ROLE_PAGE in $ROLE_PAGES; do + parse_role_variables "$ROLE_PAGE" +done +for ROLE_PAGE in $ROLE_PAGES; do + cp_role_page "$ROLE_PAGE" +done +generate_roles_toc + + +cp_single_page install +cp_single_page contribute +cp_single_page scripts From 3667635a06885a6b45320f962f8cfbde4a244d12 Mon Sep 17 00:00:00 2001 From: Code Enigma CI Date: Wed, 8 Jan 2025 12:08:18 +0000 Subject: [PATCH 183/190] GitHub Actions - updating markdown docs - --- docs/_Sidebar.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index a50bee09..f27bbbbe 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -3,6 +3,15 @@ - [Install](install) - [Usage](scripts) - [Roles](roles) + - [Exit](roles/_exit) + - [Init](roles/_init) + - ["Meta"](roles/_meta) + - [Drupal 10](roles/_meta/deploy-drupal10) + - [Drupal 7](roles/_meta/deploy-drupal7) + - [Drupal 8](roles/_meta/deploy-drupal8) + - [Matomo](roles/_meta/deploy-matomo) + - [Mautic](roles/_meta/deploy-mautic) + - [SimpleSAMLphp](roles/_meta/deploy-simplesamlphp) - [API call](roles/api_call) - [Config](roles/cache_clear) - [Drupal 7](roles/cache_clear/cache_clear-drupal7) @@ -32,17 +41,8 @@ - [MySQL backups](roles/database_backup/database_backup-mysql) - [Deploy](roles/deploy_code) - [Deploy container](roles/deploy_container) - - [Exit](roles/_exit) - - [Init](roles/_init) - [LHCI run](roles/lhci_run) - [Maintenance Mode](roles/maintenance_mode) - - ["Meta"](roles/_meta) - - [Drupal 10](roles/_meta/deploy-drupal10) - - [Drupal 7](roles/_meta/deploy-drupal7) - - [Drupal 8](roles/_meta/deploy-drupal8) - - [Matomo](roles/_meta/deploy-matomo) - - [Mautic](roles/_meta/deploy-mautic) - - [SimpleSAMLphp](roles/_meta/deploy-simplesamlphp) - [NPM](roles/npm) - [Sync roles](roles/sync) - [Database sync](roles/sync/database_sync) From a8aace549cb237dd0c324f0525edb164619fbafe Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 8 Jan 2025 13:36:01 +0100 Subject: [PATCH 184/190] Making devel PR for docs and better error handling. --- .github/workflows/ce-deploy-publish-docs.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index 063ef344..29161a56 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -51,7 +51,9 @@ jobs: # Create docs pull request - name: Create a documentation pull request - run: gh pr create --base ${{ github.event.pull_request.base.ref }} --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update" --body "**Automated pull request** created by GitHub Actions because of a documentation update." + run: | + gh pr create --base ${{ github.event.pull_request.base.ref }} --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update - ${{ github.event.pull_request.base.ref }}" --body "**Automated pull request** created by GitHub Actions because of a documentation update." || echo "No commits between ${{ github.event.pull_request.base.ref }} and docs-${{ github.event.pull_request.base.ref }} - no PR created!" + gh pr create --base devel --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update - devel" --body "**Automated pull request** created by GitHub Actions because of a documentation update." || echo "No commits between devel and docs-${{ github.event.pull_request.base.ref }} - no PR created!" env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} From 66663aee405ea9a40d1b9149bd0afbca8a627507 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 8 Jan 2025 13:36:43 +0100 Subject: [PATCH 185/190] Publish docs pr devel (#544) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Mautic updates pr 1.x (#531) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Mautic updates pr 1.x (#534) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code. * Fixing docs publishing. * Updating docs. * Publish docs pr 1.x (#536) * Fixing docs publishing. * Updating docs. * Updating name of job. * Publish docs pr 1.x (#538) * Fixing docs publishing. * Updating docs. * Updating name of job. * lockfile_behavior_bugfix (#540) * Making docs publishing changes from ce-provision in ce-deploy. * Making devel PR for docs and better error handling. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> --- .github/workflows/ce-deploy-publish-docs.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index 063ef344..29161a56 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -51,7 +51,9 @@ jobs: # Create docs pull request - name: Create a documentation pull request - run: gh pr create --base ${{ github.event.pull_request.base.ref }} --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update" --body "**Automated pull request** created by GitHub Actions because of a documentation update." + run: | + gh pr create --base ${{ github.event.pull_request.base.ref }} --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update - ${{ github.event.pull_request.base.ref }}" --body "**Automated pull request** created by GitHub Actions because of a documentation update." || echo "No commits between ${{ github.event.pull_request.base.ref }} and docs-${{ github.event.pull_request.base.ref }} - no PR created!" + gh pr create --base devel --head docs-${{ github.event.pull_request.base.ref }} --title "Documentation update - devel" --body "**Automated pull request** created by GitHub Actions because of a documentation update." || echo "No commits between devel and docs-${{ github.event.pull_request.base.ref }} - no PR created!" env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} From 8940b6abe97002b9d939004abe14da30c29462ef Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 8 Jan 2025 13:48:22 +0100 Subject: [PATCH 186/190] Documentation update - devel (#547) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Mautic updates pr 1.x (#531) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Mautic updates pr 1.x (#534) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code. * Publish docs pr 1.x (#536) * Fixing docs publishing. * Updating docs. * Publish docs pr 1.x (#538) * Fixing docs publishing. * Updating docs. * Updating name of job. * lockfile_behavior_bugfix (#540) * GitHub Actions - updating markdown docs - --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: Greg Harvey Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> Co-authored-by: Code Enigma CI --- contribute/toc.sh | 2 +- docs/_Sidebar.md | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/contribute/toc.sh b/contribute/toc.sh index 8c108c60..8dc9a1c8 100755 --- a/contribute/toc.sh +++ b/contribute/toc.sh @@ -114,7 +114,7 @@ parse_roles_toc(){ "# "*) if [ "$WRITE" = "true" ]; then TITLE=$(echo "$LINE" | cut -c 3-) - echo "$INDENT"" - [$TITLE](/$RELATIVE)" >> "$TMP_SIDEBAR" + echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" WRITE="false" fi ;; diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index a50bee09..f27bbbbe 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -3,6 +3,15 @@ - [Install](install) - [Usage](scripts) - [Roles](roles) + - [Exit](roles/_exit) + - [Init](roles/_init) + - ["Meta"](roles/_meta) + - [Drupal 10](roles/_meta/deploy-drupal10) + - [Drupal 7](roles/_meta/deploy-drupal7) + - [Drupal 8](roles/_meta/deploy-drupal8) + - [Matomo](roles/_meta/deploy-matomo) + - [Mautic](roles/_meta/deploy-mautic) + - [SimpleSAMLphp](roles/_meta/deploy-simplesamlphp) - [API call](roles/api_call) - [Config](roles/cache_clear) - [Drupal 7](roles/cache_clear/cache_clear-drupal7) @@ -32,17 +41,8 @@ - [MySQL backups](roles/database_backup/database_backup-mysql) - [Deploy](roles/deploy_code) - [Deploy container](roles/deploy_container) - - [Exit](roles/_exit) - - [Init](roles/_init) - [LHCI run](roles/lhci_run) - [Maintenance Mode](roles/maintenance_mode) - - ["Meta"](roles/_meta) - - [Drupal 10](roles/_meta/deploy-drupal10) - - [Drupal 7](roles/_meta/deploy-drupal7) - - [Drupal 8](roles/_meta/deploy-drupal8) - - [Matomo](roles/_meta/deploy-matomo) - - [Mautic](roles/_meta/deploy-mautic) - - [SimpleSAMLphp](roles/_meta/deploy-simplesamlphp) - [NPM](roles/npm) - [Sync roles](roles/sync) - [Database sync](roles/sync/database_sync) From 733d2dfbbee9dde1b0eecc2bd095cd145558d6f7 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 8 Jan 2025 14:52:10 +0100 Subject: [PATCH 187/190] Making job skipping work properly. --- .github/workflows/ce-deploy-publish-docs.yml | 25 +++++++++++++++----- 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index 29161a56..92540134 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -7,16 +7,29 @@ on: branches: - 1.x workflow_dispatch: +# Set target docs branch name +env: + docs_branch: docs-${{ github.event.pull_request.base.ref }} jobs: - # Set the job key. The key is displayed as the job name - # when a job name is not provided + # See https://stackoverflow.com/a/74378072 + set-docs-branch: + name: Make docs branch name available to publish-docs job + runs-on: ubuntu-latest + outputs: + docs_branch: ${{ steps.init.outputs.docs_branch }} + + steps: + - name: Make environment variables global + id: init + run: | + echo "docs_branch=${{ env.docs_branch }}" >> $GITHUB_OUTPUT + publish-docs: + name: Publish the ce-deploy docs to GitHub # Only run the job if it is not coming from a documentation branch - if: ${{ github.event.pull_request.head.ref != 'docs-${{ github.event.pull_request.base.ref }}' }} - # Name the Job - name: Publish ce-deploy documentation - # Set the type of machine to run on + needs: set-docs-branch + if: ${{ github.event.pull_request.head.ref != needs.set-docs-branch.outputs.docs_branch }} runs-on: ubuntu-latest # Use our ce-dev Debian base container From e9927164eb111ce3ee038556b61ec9b425922681 Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 8 Jan 2025 14:52:55 +0100 Subject: [PATCH 188/190] Publish docs pr devel (#550) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Bug fixes pr 1.x (#278) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Bug fixes pr 1.x (#282) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Bug fixes pr 1.x (#284) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Bug fixes pr 1.x (#287) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Adding verbose output for drush. (#289) * Adding container push and build code. (#300) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Ecs deployments pr 1.x (#314) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * db_import_speed_up (#291) * db_import_speed_up * db_import_speed_up_fixed * db_import_speed_up_fix2 * db_import_speed_up_fix3 * db_import_speed_up_fix4 * db_import_speed_up_fix5_and_gzip * db_import_speed_up_fix6 * db_import_speed_up_fix7 * db_import_speed_up_fix_typo * db_import_speed_up_fix8 * db_import_speed_up_fix9 * db_import_speed_up_refactoring * Ecs deployments pr 1.x (#317) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * fix_typo (#320) * fix_typo * fix_typo * Ecs deployments pr 1.x (#322) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Ecs deployments pr 1.x (#325) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Ecs deployments pr 1.x (#329) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * Ecs deployments pr 1.x (#331) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * Bug fixes pr 1.x (#334) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Bug fixes pr 1.x (#336) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * default_to_gzip (#338) * update_for_database_names_handling (#340) * update_for_database_names_handling_fix (#342) * Adding syncing for feature branches. (#346) * Feature branching pr 1.x (#348) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Feature branching pr 1.x (#350) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * Improving db handling and fixing adding databases later. (#358) * Improving db handling and fixing adding databases later. * Bad spacing caught by linter. * Making an explicit debug entry if we skip database population. * Allowing override of 'handling' on a database-by-database basis. * Tab spacing errors caught by linter. * Syntax error in my debug. * Being more verbose in task naming. * Enforcing bash as the interpretor for shell. * Adding new database handling to dump and none types. * Accidentally doubled the args key on a task. * Variable naming error. * Add -w flag to grep to match only the whole database name when checking for db existence. * Bug fixes pr 1.x (#366) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Bug fixes pr 1.x (#368) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Bug fixes pr 1.x (#372) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Bug fixes pr 1.x (#374) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Bug fixes pr 1.x (#375) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * r66858-adding-variable-to-skip-long-running-tasks (#352) Co-authored-by: Matej Stajduhar * Fixing documentation for opcache role. (#354) * r67016-changing-path-for-database_backup-task (#356) * r67016-changing-path-for-database_backup-task * Slight refactor to use a directory we can safelist with rkhunter. * Docs update. * Removing rogue extra line. --------- Co-authored-by: Matej Stajduhar Co-authored-by: gregharvey * 67412 callback plugin fail no hosts pr 1.x (#377) * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Removing obsolete SOPS plugin and adding new callback to fail when no hosts match. * Bug fixes pr 1.x (#379) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Bug fixes pr 1.x (#381) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Bug fixes pr 1.x (#383) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * Bug fixes pr 1.x (#386) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Bug fixes pr 1.x (#388) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Bug fixes pr 1.x (#391) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Bug fixes pr 1.x (#393) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Ecs deployments pr 1.x (#397) * Adding container push and build code. * Fixing some minor issues. * Docs updates and template flexibility improvement. * Creating separate registry name for handling docker push. * Trying to get the registry name right. * Adding AWS stack components to ECS builds. * Force a docker logout. * Need to delegate docker commands to the controller. * Trying deleting the credentials file instead. * Fixing var name in subnet.yml. * Missing some ACM variables. * Allowing us to set wait timeout for TG creation. * Uncommenting cpu and memory vars for ECS with Fargate. * Missed a variable in ELB creation. * Couple of linting fixes. * Force building and tagging of containers as an option. * Improving docs for deploy_container role. * Providing container description example for ECS. * Adding 'force' option to docker builds. * Revert "Adding 'force' option to docker builds." This reverts commit 5574faa182f023c20ec34ae6dc7125c92b4c7dcb. * Adding options to force ECS component refreshes. * Allowing users to toggle public IP and execute command mode. * The ALB needs to be on public subnets, not the same private ones as the cluster. * Updating documentation. * Updating ECS docs. * New deploy_container destroy action. * Documentation update. * Adding a note to docs about the container destroy action. * Bug fixes pr 1.x (#400) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Bug fixes pr 1.x (#402) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * r67637-terminate-sessions-keeping-loop-device-busy (#405) * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * r67637-terminate-sessions-keeping-loop-device-busy * Avoiding-cache-clear-warning-for-site-variable (#415) Co-authored-by: Matej Stajduhar * additional check and debug task (#417) * additional check and debug task * additional check and debug task * additional check and debug task * additional_checks_and_moving_debug_task_to_simplify_diagnostics * r65896-cache-clear-path (#422) * r65896-cache-clear-dir * r65896-cache-clear-dir * grouping squashfs tasks * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-clear-dir * r65896-cache-container-table-truncate (#434) * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * r65896-cache-container-table-truncate * roles/_init/defaults/main.yml * r65896-cache-container-table-truncate-change-tasks-order (#439) * Bug fixes pr 1.x (#441) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Adding a pause if a replica is declared, so the replica can catch up with the writer. (#443) * Db replica support pr 1.x (#445) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Db replica support pr 1.x (#447) * Adding a pause if a replica is declared, so the replica can catch up with the writer. * Making read replica support more robust. * Ensuring mysql backup takes no action on replicas. * Suppressing backup jobs for replica databases. * Minor docs update. * 448: Drupal 10 default.settings.php for multi-site MR 1.x (#449) * 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context. * 448: Corrected linting issue. * 448: Corrected code style issue. * 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups. * 448: Added a .j2 to the end of the default.settings.php template reference. --------- Co-authored-by: Philip Norton * New StatusCake check handling role. (#454) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Statuscake maintenance role pr 1.x (#456) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Statuscake maintenance role pr 1.x (#458) * New StatusCake check handling role. * Updating maintenance mode defaults and options. * Following StatusCake task naming conventions. * Missed the body_format param in StatusCake URI call. * Ensuring we pass the test ID as a string to the StatusCake URI call. * Fixing namespaces. * Stashing the original deploy_path in initial_deploy_path for squashfs builds. * Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode. * We need the path to the previous build for maintenance mode online. * Changing order of tasks so previous_build_number is set earlier. * We also need a previous_drush_bin. * Adding extra healthcheck options for ECS services. (#463) * Adding behaviour to suspend autoscaling on deploy. (#465) * Ecs healthcheck vars pr 1.x (#467) * Adding extra healthcheck options for ECS services. * Renaming scale up/down to in/out to avoid confusion. * Spotted annoying typo. * Fixing MySQL servers with unix socket. * Bad var name. * Extending MySQL Unix socket support. * Bug fixes pr 1.x (#469) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Lock file pr 1.x (#471) * Adding pre-flight checks and lock file behaviour to ce-deploy. * Moving lock file paths to variables. * Bug fixes pr 1.x (#476) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Adding a new Drupal 10 meta role. (#478) * Adding a new Drupal 10 meta role. * Bad copy/paste in docs. * r69347-db-import-from-deploy-server (#480) * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * r69347-db-import-from-deploy-server * typo fixes and tidying up * Bug fixes pr 1.x (#486) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * 472 Cache clearing for Symfony based projects (#473) * 472 Ensure Mautic cache is cleared on every host * 472 Add Symfony cache clear task * 472 Make console path configurable * 472 Fix command variable * Feature branching pr 1.x (#488) * Adding syncing for feature branches. * Cannot loop over import_role, changing for include_role. * Adding ability to specify an exact filename for a settings template for Drupal 8 and above. * First pass at NGINX vhost handling in ce-deploy. * Adding in the SSL role from ce-provision. * Fixing role paths and ensuring NGINX ssl.yml is available. * Fixing NGINX role location. * Tweaking location of domain.yml so it loads the templates correctly. * additional_symlink_task_fix_for_squashfs (#494) * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fix_for_squashfs * additional_symlink_task_fixes * additional_symlink_task_fixes * fix cachetool url (#500) * strip sandbox mode line from sqldump (#502) * Updating-crons-mailto (#506) * Updating-crons-mailto * Forgot-variable-drupal8 * Updating-cron-path-fixing-mailto (#508) * Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting * Bug fixes pr 1.x (#530) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Mautic updates pr 1.x (#531) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Mautic updates pr 1.x (#534) * Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code. * Fixing docs publishing. * Updating docs. * Publish docs pr 1.x (#536) * Fixing docs publishing. * Updating docs. * Updating name of job. * Publish docs pr 1.x (#538) * Fixing docs publishing. * Updating docs. * Updating name of job. * lockfile_behavior_bugfix (#540) * Making docs publishing changes from ce-provision in ce-deploy. * Publish docs pr 1.x (#542) * Fixing docs publishing. * Updating docs. * Updating name of job. * Making docs publishing changes from ce-provision in ce-deploy. * Making devel PR for docs and better error handling. * Publish docs pr 1.x (#545) * Fixing docs publishing. * Updating docs. * Updating name of job. * Making docs publishing changes from ce-provision in ce-deploy. * Making devel PR for docs and better error handling. * GitHub Actions - updating markdown docs - (#543) Co-authored-by: Code Enigma CI * Documentation update - 1.x (#546) * Merging 1.x down into devel (#269) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Target host checking pr devel (#270) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Sync enhancements pr devel (#272) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Bug fixes pr devel (#274) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Revert "Bug fixes pr devel (#274)" (#276) This reverts commit 3e2977500ee9fb68c8e5b01e836e07718549ab66. * Bug fixes pr devel (#277) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role.… * Making job skipping work properly. --------- Co-authored-by: EmlynK Co-authored-by: Dionisio Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> Co-authored-by: Matej Štajduhar <30931414+matej5@users.noreply.github.com> Co-authored-by: Matej Stajduhar Co-authored-by: Philip Norton Co-authored-by: Philip Norton Co-authored-by: Chris Maiden Co-authored-by: nfawbert <62660788+nfawbert@users.noreply.github.com> Co-authored-by: drazenCE <140631110+drazenCE@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Code Enigma CI --- .github/workflows/ce-deploy-publish-docs.yml | 25 +++++++++++++++----- 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ce-deploy-publish-docs.yml b/.github/workflows/ce-deploy-publish-docs.yml index 29161a56..92540134 100644 --- a/.github/workflows/ce-deploy-publish-docs.yml +++ b/.github/workflows/ce-deploy-publish-docs.yml @@ -7,16 +7,29 @@ on: branches: - 1.x workflow_dispatch: +# Set target docs branch name +env: + docs_branch: docs-${{ github.event.pull_request.base.ref }} jobs: - # Set the job key. The key is displayed as the job name - # when a job name is not provided + # See https://stackoverflow.com/a/74378072 + set-docs-branch: + name: Make docs branch name available to publish-docs job + runs-on: ubuntu-latest + outputs: + docs_branch: ${{ steps.init.outputs.docs_branch }} + + steps: + - name: Make environment variables global + id: init + run: | + echo "docs_branch=${{ env.docs_branch }}" >> $GITHUB_OUTPUT + publish-docs: + name: Publish the ce-deploy docs to GitHub # Only run the job if it is not coming from a documentation branch - if: ${{ github.event.pull_request.head.ref != 'docs-${{ github.event.pull_request.base.ref }}' }} - # Name the Job - name: Publish ce-deploy documentation - # Set the type of machine to run on + needs: set-docs-branch + if: ${{ github.event.pull_request.head.ref != needs.set-docs-branch.outputs.docs_branch }} runs-on: ubuntu-latest # Use our ce-dev Debian base container From 961cc8fa14d1780921a1474d3d53050dc2a45dae Mon Sep 17 00:00:00 2001 From: Greg Harvey Date: Wed, 8 Jan 2025 14:56:36 +0100 Subject: [PATCH 189/190] Adding the opening slash back in. --- contribute/toc.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contribute/toc.sh b/contribute/toc.sh index 8dc9a1c8..8c108c60 100755 --- a/contribute/toc.sh +++ b/contribute/toc.sh @@ -114,7 +114,7 @@ parse_roles_toc(){ "# "*) if [ "$WRITE" = "true" ]; then TITLE=$(echo "$LINE" | cut -c 3-) - echo "$INDENT"" - [$TITLE]($RELATIVE)" >> "$TMP_SIDEBAR" + echo "$INDENT"" - [$TITLE](/$RELATIVE)" >> "$TMP_SIDEBAR" WRITE="false" fi ;; From de01f0f13dfe68c281a778c754641ea9d9d899f7 Mon Sep 17 00:00:00 2001 From: Code Enigma CI Date: Mon, 13 Jan 2025 17:07:04 +0000 Subject: [PATCH 190/190] GitHub Actions - updating markdown docs - 2025-01-08T14:01:54Z --- docs/_Sidebar.md | 88 ++++++++++++++++++++++++------------------------ 1 file changed, 44 insertions(+), 44 deletions(-) diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md index f27bbbbe..44ec67bc 100644 --- a/docs/_Sidebar.md +++ b/docs/_Sidebar.md @@ -3,47 +3,47 @@ - [Install](install) - [Usage](scripts) - [Roles](roles) - - [Exit](roles/_exit) - - [Init](roles/_init) - - ["Meta"](roles/_meta) - - [Drupal 10](roles/_meta/deploy-drupal10) - - [Drupal 7](roles/_meta/deploy-drupal7) - - [Drupal 8](roles/_meta/deploy-drupal8) - - [Matomo](roles/_meta/deploy-matomo) - - [Mautic](roles/_meta/deploy-mautic) - - [SimpleSAMLphp](roles/_meta/deploy-simplesamlphp) - - [API call](roles/api_call) - - [Config](roles/cache_clear) - - [Drupal 7](roles/cache_clear/cache_clear-drupal7) - - [Drupal 8](roles/cache_clear/cache_clear-drupal8) - - [Matomo](roles/cache_clear/cache_clear-matomo) - - [Opcache](roles/cache_clear/cache_clear-opcache) - - [Symfony](roles/cache_clear/cache_clear-symfony) - - [ce-dev](roles/ce_dev) - - [ce-dev prebuild](roles/ce_dev/ce_dev_prebuild) - - [CLI Tools](roles/cli) - - [Drush](roles/cli/cachetool) - - [Drush](roles/cli/drush) - - [Composer](roles/composer) - - [Config](roles/config_generate) - - [Drupal 7](roles/config_generate/config_generate-drupal7) - - [Drupal 8](roles/config_generate/config_generate-drupal8) - - [Drupal 8](roles/config_generate/config_generate-matomo) - - [Mautic](roles/config_generate/config_generate-mautic) - - [Cron](roles/cron) - - [Database backup cron task](roles/cron/cron_database_backup) - - [Database backup cron task - MySQL](roles/cron/cron_database_backup/cron_database_backup-mysql) - - [Drupal 7 cron](roles/cron/cron_drupal7) - - [Drupal 8 cron](roles/cron/cron_drupal8) - - [Database backup cron task - MySQL](roles/cron/cron_matomo) - - [Mautic cron tasks](roles/cron/cron_mautic) - - [Data backups](roles/database_backup) - - [MySQL backups](roles/database_backup/database_backup-mysql) - - [Deploy](roles/deploy_code) - - [Deploy container](roles/deploy_container) - - [LHCI run](roles/lhci_run) - - [Maintenance Mode](roles/maintenance_mode) - - [NPM](roles/npm) - - [Sync roles](roles/sync) - - [Database sync](roles/sync/database_sync) - - [Database sync - MySQL](roles/sync/database_sync/database_sync-mysql) + - [Exit](/roles/_exit) + - [Init](/roles/_init) + - ["Meta"](/roles/_meta) + - [Drupal 10](/roles/_meta/deploy-drupal10) + - [Drupal 7](/roles/_meta/deploy-drupal7) + - [Drupal 8](/roles/_meta/deploy-drupal8) + - [Matomo](/roles/_meta/deploy-matomo) + - [Mautic](/roles/_meta/deploy-mautic) + - [SimpleSAMLphp](/roles/_meta/deploy-simplesamlphp) + - [API call](/roles/api_call) + - [Config](/roles/cache_clear) + - [Drupal 7](/roles/cache_clear/cache_clear-drupal7) + - [Drupal 8](/roles/cache_clear/cache_clear-drupal8) + - [Matomo](/roles/cache_clear/cache_clear-matomo) + - [Opcache](/roles/cache_clear/cache_clear-opcache) + - [Symfony](/roles/cache_clear/cache_clear-symfony) + - [ce-dev](/roles/ce_dev) + - [ce-dev prebuild](/roles/ce_dev/ce_dev_prebuild) + - [CLI Tools](/roles/cli) + - [Drush](/roles/cli/cachetool) + - [Drush](/roles/cli/drush) + - [Composer](/roles/composer) + - [Config](/roles/config_generate) + - [Drupal 7](/roles/config_generate/config_generate-drupal7) + - [Drupal 8](/roles/config_generate/config_generate-drupal8) + - [Drupal 8](/roles/config_generate/config_generate-matomo) + - [Mautic](/roles/config_generate/config_generate-mautic) + - [Cron](/roles/cron) + - [Database backup cron task](/roles/cron/cron_database_backup) + - [Database backup cron task - MySQL](/roles/cron/cron_database_backup/cron_database_backup-mysql) + - [Drupal 7 cron](/roles/cron/cron_drupal7) + - [Drupal 8 cron](/roles/cron/cron_drupal8) + - [Database backup cron task - MySQL](/roles/cron/cron_matomo) + - [Mautic cron tasks](/roles/cron/cron_mautic) + - [Data backups](/roles/database_backup) + - [MySQL backups](/roles/database_backup/database_backup-mysql) + - [Deploy](/roles/deploy_code) + - [Deploy container](/roles/deploy_container) + - [LHCI run](/roles/lhci_run) + - [Maintenance Mode](/roles/maintenance_mode) + - [NPM](/roles/npm) + - [Sync roles](/roles/sync) + - [Database sync](/roles/sync/database_sync) + - [Database sync - MySQL](/roles/sync/database_sync/database_sync-mysql)